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F  Hi  w.vr.) 


The  staggering  costs  of  modern  weapon  systems  oa  ye  '>purreo  t'1° 
search  for  more  oconomi  ca  1  methods  if  weapon  procurement .  ,/ar'y  d-  -e~$e 
establishment  critics  nave  touted  increased  reliance  on  con>:'*  :  •  ■ 
bidding  and  dual -source  production  as  methods  to  reduce  costs. 
Lieutenant  Colonel  Hampton's  study  addresses  this  important  subject. 

£o l on«4  - Hampton  proposes  a  discounted  cash  fl  >w  investment  mcHe : 
the  appropriate  framework  for  analyzing  the  cost -effect  i  veres of 
developing  a  second  source  during  the  production  phase  of  a  major  weapon 
system  acquisition.  He  outlines  the  factors  and  impacts  that  or- 

considered  in  second-source  analysis  and  reviews  s'-  .'11 

program-specific  analyses  to  determine  if  all  factors  and  impj"'  were 
considered.  Finally,  he  proposes  a  discounted  -a  so  flow  i  n  new* 

model  for  second-source  production  analyses  to  help  ;n  dec i d1 -c  tne 
probabi  1  i  ty  that  government  investment  in  nonrecurring  cost  w’il  r>e 
recovered. 

We  believe  Colonel  Hampton's  findings  a  rv>  important  in  oir 
continuing  attempts  to  rationalize  weapon  systems  procurement.  Alfoougn 
everyone  may  not  agree  with  his  proposals.  Colonel  Hampton's  s^riy 
provides  a  significant  challenge1  to  those  who  have  approached  tne 
subject  in  a  less  thorough  and  systematic  manner. 


DON  A 10  0.  STEVENS,  Colonel,  'JSAF 
Vice  Commander 

Center  for  Aerospace  Doctrine, 
Research,  and  Education 
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My  original  charge  from  11  r  i  y  (Ion  Dan  Goran,  AF  SC  DCS  'Compt-o e-  < 
was  to  develop  for  the  cognoscenti  an  alyorithia  to  us°  in  analyyi'v:  toe 
effect  that  tne  introduction  of  competition  through  ^cnid-vj  /c  ' 
would  have  on  the  cost  profile  of  an  acquisition  of  a  weapon  wsV-'  .  f 
introduced  into  the  production  phase.  It  was  my  understanding  tn,;f  foe 
algorithm  would  specify  the  parameters  to  he  applied  against  gr 
specific  data  in  a  mecnanistic  fashion,  crank inu  out  the  anew*--  -r 
whether  it  was  cost-effective  to  develop  a  second  source  and  s  n- 
sequently  budgeting  for  the  compe*- i  t  i  v<>  program.  And  General  in -an  "id 
every  right  to  expect  me  to  develop  the  model.  Several  pr--.--;  ;<  i  jr> 
models  currently  exist  in  the  literature  and  he  wanted  to  know  nad 

the  most  valid i ty  . 

I  have  not  accomplished  my  original  tasking!  After  rev  .-w  i  *  ne 
1  i  terature--wh i ch  consisted  of  both  predictive  models  and  critiques  of 
the  models--I  concluded  that  a  mecnanistic  algorithm  caul  1  not  *>-> 
developed.  1  concluded  that  the-'e  are  simply  too  many  variables  boat 
can  affect  the  magnitude  of  savings  or  added  cost  on  a  given  acquisition 
to  formulate  a  mathematical  model.  And  each  critique  that  I  reviewed 
drew  the  same  conclusion! 

The  fact  that  an  algorithm  cannot  be  formulated  does  not  inaxe  tne 
need  for  a  decision  analysis  disappear.  Acquisition  strategies  -’ust 
still  be  planned  and  budgets  developed.  In  lieu  of  an  algorithm,  1  m 
proposing  th.it  we  view  the  development  of  a  second  source  j;  an 
investment  decision,  made  with  the  goal  of  obtaining  a  return  on  ur 
i n vestment . 

My  review  of  the  literature  revealed  two  widespread  misunderstand¬ 
ings  relating  to  compet i t i on--bof h  in  the  legislative  and  executive 
branches.  First  was  a  confusion  between  the  processes  of  forma’ 
advertising,  negotiation,  and  competition.  Many  have  equated  for -a1 
advertising  with  competition,  ignoring  competitive  negotiations  as  a 
means  of  competition.  A  second  mi sp.-rrnpt ion  exists  that  there  is  an 
empirically  verifier!  average  savings  that  results  when  sole-source 
production  programs  are  competed.  Chapter  1  explores  t.nese 
misperceptions  for  the  cognoscenti  and  layman  alike  because  they  are  the 
foundation  for  many  i 1  I -concei vod  actions  within  both  the  legislative 
and  executive  branches. 

Chapters  2  thru  S  are  written  for  the  cognoscenti,  the  a cgu i s i t 1  v 
professionals  that  must  make  the  decision  to  second-source  or  not,  avd 
subsequently  implement  the  decision.  Chapter  2  outlines  what  factors 
and  impacts  must  be  considered  in  a  second-source  analysis.  Chantv'  ' 
reviews  published  studies  of  past,  competitive  procurements.  In 
determines  the  method  and  data  used  to  arrive  at  study  conclusions  and 
provides  a  judgement-  of  the  validity  of  study  conclusions.  ''nap to"  - 
reviews  l)0D  progra  o-speci  f  i  c  analys  >*.  to  determine  if  all  f  ioo'-s  r.n 
impacts  delineated  in  chapter  2  w ■  * r - >  onsidereU.  'oapter  i>  prunes  ■  a 
discounted  cash  flow  investment  -,o  |e  ;  tor  iso  in  or  'petition  anal  ,-cm, . 


!  he  3  been  p<i rt  i cu  1  a r] y  fortunate  to  have  this  tour  as  the  Air 
Force  Systems  Command  'desearch  Fellow  for  1M8J-M4.  It  contributed  to  my 
professional  education  as  an  attendee  at  the  Air  'war  College.  And  my 
'"esearch  project  enabled  me--th rough  an  in-depth  review  of  several  Army, 
Navy  and  Air  Force  programs,  and  my  travels  to  all  AFSC  buying 
di  vi  s i ons--to  gain  a  better  understanding  and  appreciation  for  the 
complex  interaction  the  program  manager  must  orchestrate  between  the 
contractor,  engineers,  program  control,  comptroller,  and  contracting 
personnel  among  others. 

The  review  of  studies  and  cr i t iques- -a  1  1  of  which  were  either 
accomplished  in-house  or  funded  under  government  cont ract -- revea 1 ed  a 
disconnect  between  researchers  and  practitioners  as  the  critiques  of 
published  predictive  models  raised  the  flag  of  caution  on  the  use  of 
various  studies.  Yet  the  use  of  study  results  in  program-spec i f  i c 
analyses  show  how  quickly  the  key  assumptions  and  cautions  of  study 
conclusions  fall  by  the  wayside,  and  the  critiques  ignored,  as 
p ract i t i oners  inappropriately  applied  study  conclusions  by  grabbing  any 
"authoritative"  source  to  accomplish  their  analyses. 

I  It  also  seems  that  the  Department  of  Defense  can  be  its  own  worst 

enemy.  We  provide  data,  either  knowingly  or  naively,  about  savings 

estimates  that  are  not  accurate.  And  we  do  not  take  action  to  correct 
existing  misperceptions.  0MB  and  the  Congress  then  take  actions  based 
on  the  i nappropri ate  information.  We  must  somehow  stop  this  vicious 
cycle. 
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CHAPTER  1 


INTRODUCTION  TO  SECOND-SOURCING 

Capitalism  has  given  the  United  States  the  highest  standa"'  >■ 
living  ever  achieved  by  any  society  in  the  world.  The  basic  tenets  of 
capitalism  include  private  ownership  of  the  factors  of  production,  with 
prices,  production  and  the  distribution  of  goods  being  determine  r. 
competition  between  enterprises  in  a  free-market  system.  These  tenets 
stimulate  technological  innovation  and  drive  prices  down  as  more 
entrepreneurs  enter  the  marketplace  in  anticipation  of  earning  a  T-of it, 
a  return  on  their  investment  in  the  factors  of  production,  and  a  '■award 
for  their  risk  taking.  Vet,  in  fiscal  year  (FY)  1982,  58.9  percent  of 
Department  of  Defense  (DOD)  procurement  dollars  and  65.8  percent  of  Air 
Force  procurement  dollars  were  expended  nonconpet i t i vely .  And  of  the 
total  FY  1982  procurement  dollars  expended,  25.6  percent  of  DOT  dollars 
and  48.6  percent  of  Air  Force  dollars  were  noncompetitive  follow-on 
production  contracts.  Because  of  these  apparently  high  percentages  ana 
the  high  costs  of  today's  weapons  systems  both  in  absolute  cost  and  the 
overruns  that  are  incurred,  the  executive  and  legislative  branches  are 
applying  pressure  to  increase  competition  in  the  production  phase  of  tne 
weapons  acquisition  process  with  the  hope  of  reducing  system  costs  and 
improving  system  technical  performance. 


Resear c h  ( ) u_e_s t io n s 

This  study  explores  dual-sourcing  as  a  means  of  increasing 
production  competition  and  provides  an  analytical  framework  for 
assistance  in  making  dual -sourcing  decisions  and  budgeting  for  weapon 
systems  that  will  be  dual -sourced.  Dua 1 -sourci ng  during  production 
exists  when  two  or  more  firms  are  capabl°  of  furnishing  government 
requirements  with  contract  award  determined  by  price,  either  t’nrojgn 
split-awards  over  time  or  a  competitive  buy-out. 

To  accomplish  this  objective,  the  primary  research  quest’on 
guiding  the  study  was: 

What  analytical  framework  is  appropriate  for  estimating,  a  priori, 
the  costs  and  benefits  of  dua 1 -sourci ng  weapons  in  production? 

In  answering  the  primary  research  question,  three  subsidiary 
research  questions  had  to  be  answered: 

What  factors  must  be  considered  when  measuring  dual -source  savings 
by  quantifying  its  costs  and  benefits? 

What  ex  post  (after  the  fact)  academic  and  consultant  studies  •\a /e 
been  accomplished  quantifying  the  rusts  and  benefits  of  dua  1  -sou  rc  i n , 
and  what  predictive  methods  have  tne/  developed  for  ex  ante  (before, 
fact)  use? 


i 


How  have  military  system  program  offices  and  comptrollers  estimated 
ex  ante  the  costs  and  benefits  of  dual -sourcing? 

Chapter  2  addresses  the  first  subsidiary  research  question  and 
identifies  cost  and  benefit  factors  that  must  be  quantified  to  estimate 
the  net  cost  impact  associated  with  dua 1 -sourci ng.  Chapter  3  reviews 
and  critiques  previous  studies  by  both  the  academic  and  consultant 
communities  which  attempt  to  quantify  the  savings  on  weapons  systems 
that  have  been  dual -sourced,  thereby  answering  the  second  subsidiary 
question.  Chapter  4  treats  the  third  subsidiary  question  and  reviews 
internal  government  studies  relating  to  the  dua 1 -sourci ng  decisions  for 
a  sample  of  programs.  Chapter  5  discusses  a  proposed  analytical 
framework  for  making  initial  dual-sourcing  decisions  and  subsequent 
dual -sourcing  budgeting  decisions.  Chapter  5  also  discusses  how  the 
framework  should  be  used  in  000  decision-making  and  the  implications  of 
the  findings  for  acquisition  management  personnel.  First,  however,  the 
balance  of  this  chapter  differentiates  competition  from  other 
acquisition  concepts,  reviews  the  stages  of  the  acquisition  process, 
discusses  recent  pressures  to  increase  production  competition,  and 
briefly  surveys  the  competition  literature.  It  also  identifies  the 
reasons  why  a  high  percentage  of  Air  Force  procurement  dollars  are  spent 
noncompet i ti vely  and  sets  forth  the  circumstances  under  which  costs  can 
be  avoided  by  dua 1 -sou rci ng  weapon  systems. 


Compet i  t_i_on_  and_  Formal  Advertising 


Maintaining  competition  in  the  acquisition  process  by  retaining  two 
or  more  viable  producers  as  long  as  is  economically  beneficial  is  the 
goal  of  Department  of  Defense  procurements* ;  formally  advertised 
procurement  is  the  preferred  method  to  maintain  competition. ^  Here  it 
is  useful  to  draw  a  distinction  between  formal  advertising  and 
competition  because  sometimes  there  is  confusion  in  the  usage  of  the 
terns . 


Formal  advertising  and  negotiation  are  two  concepts  grounded  in 
public  law.  Formal  advertising  is  a  procedure  whereby  the  government 
develops  detailed  specifications  for  a  requirement,  publicly  issues  an 
invitation  for  bid  ( I F  B ) ,  provides  adequate  time  for  bids  to  be 
submitted,  and  specifies  an  exact  time  and  place  at  which  the  bids  will 
be  publicly  opened.  Rids  are  evaluated  as  to  compliance  with  the  strict 
instructions  of  the  I FB  and  the  contract  is  awarded  to  the  lowest 
responsive  responsible  bidder.  This  is  in  contrast  to  the  concept  of 
negotiation,  which  is  defined  in  law  as  any  method  other  than  formal 
advertising.  Negotiation  is  more  flexible  than  formal  advertising  since 
ongoing  discussions  regarding  the  project's  technical  and  cost 
parameters  can  take  place  prior  to  contract  award.^ 


Competition  can  exist  using  pither  the  formal  advertising  or 
negotiated  procedures.  Figure  1-1  illustrates  the  percentage  of  dollar 
awards  to  competitive  and  noncompetitive  procurements.  Competitive 
procurements  include  both  formally  advertised  and  competitively 
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negotiated  contracts  and  account  for  34.3  percent  of  dollar  awards. 
Noncompetitive  awards  are  those  where  only  one  source  is  solicited  or 
the  award  is  made  noncompetitively  as  a  follow-on  to  either  technical  or 
design  competition. 

Semantic  confusion  often  results  in  the  error  of  equatiny 
competition  with  only  formally  advertised  actions.  For  example,  a  1978 
Rand  study  quotes  Senator  William  Rroxmire:  "Of  all  TOOl)]  procurement, 
only  11  percent  is  competitive."^  What  the  senator  reported  was  only 
the  formally  advertised  percentage  of  procurement  dollars.  And  in  1983 
Senator  David  Pryor  stated  "It  is  nothing  short  of  shocking  to  hear  that 
competition  at  the  Pentagon  is  limited  to  only  6  percent  of  total 
defense  contract  dollars."^  Again,  as  can  be  seen  from  figure  1-1,  he 
also  has  equated  competition  with  formal  advertising. 


W_e_ap_ons  Acqui  si  t  i  on  Process  and  Competi  tion 

Four  separate  stages  characterize  the  weapons  acquisition  process: 
concept  exploration,  demonstration  and  validation,  full-scale 
engineering  development,  and  production  and  deployment  of  the  system. 
One  must  have  a r.  understanding  of  this  process  and  how  competition  is 
generated  at  each  stage  in  order  to  (1)  understand  why  there  is  a  high 
percentage  of  noncompetitive  contract  awards,  and  (2)  why  there  are 
increasing  pressures  to  develop  competition  in  systems  procurements 

The  first  three  stages--concept  exploration,  demonstration  and 
validation,  and  full-scale  engineering  development  (FSEI) )--compri se  the 
research  and  development  (R  &  D)  phase  of  the  acquisition  process.  In 
the  concept  exploration  stage,  mission  needs  or  requirements  are 
identified  and  potential  alternate  systems  design  concepts  to  satisfy 
identified  needs  are  evaluated.  Performance  and  mission  envelopes 
'capabilities)  and  operational  concepts  are  committed  to  paper  and 
sometimes  prototype  hardware  is  developed  during  the  second 
(demonstration  and  validation)  stage  when  the  best  technical  approach  is 
selected.  This  is  followed  by  the  FSED  stage  during  which  firm 
engineering  drawings  are  drafted,  production  processes  and  procedures 
developed,  and  engineering  data  lists  compiled.  Finally,  the  production 
and  deployment  of  a  weapon  system  is  accomplished  from  the  detailed 
design  data  developed  during  the  R  &  D  stage. 

Figure  1-2  nominally  depicts  the  budget  profile  of  a  typical  weapon 
system  acquisition  by  process  staye  over  its  program  life.'7  As  each 
stage  of  the  process  is  accomplished,  technical  and  cost  uncertainty  is 
progressively  reduced.  This  results  from  the  establishment  of  the 
technical  performance  baseline  of  the  system  and  a  closer  definition  of 
design  characteristics  and  production  costs  during  each  successive 
stage.  For  example,  during  the  concept  exploration  stage,  different 
systems  concepts  such  as  missiles  and  bombers  vie  for  selection  to 
accomplish  a  required  mission.  Alternate  subsystems  for  a  selected 
system  are  generally  evaluated  in  the  demonstration  and  validation 
stage.  During  full-scale  engineering  development,  design  changes  can  be 
made  to  the  selected  subsystems.  And  engineering  changes  are  the  only 
changes  usually  made  in  a  system  during  the  production  stage. ^ 
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We  will  now  briefly  review  the  evolution  of  the  contractor 
selection  process  as  it  relates  to  the  acquisition  process  in  order  to 
identify  factors  that  have  led  to  a  significant  percentage  of 
noncompetitive  DOD  contracts  being  awarded.0 


Evolution  of  Sol  e-Source  Procurements 

Before  the  turn  of  the  century,  formal  advertising  was  the  most 
frequently  used  procurement  method.  In-house  arsenals  developed 
detailed  specifications,  which  allowed  private  enterprise  to  bid  firm 
prices  against  the  detailed  specification  package. 

Weapons  gradually  became  increasingly  complex  and  the  system  of 
in-house  arsenals  was  partially  dismantled  after  the  turn  of  the 
twentieth  century.  Therefore,  with  no  definitive  speci  fi  cati  ons--for 
most  aircraft  procurements  for  example-- the  government  negotiated 
contracts  on  a  sole-source  basis.  In  1Q?6,  in  an  effort  to  stimulate 
both  the  technical  development  of  aircraft  and  competition  in  their 
procurement.  Congress  passed  the  Aircraft  Production  Act.^  This  act 
authorized  design  competition  in  the  procurement  of  aircraft  and  the 
purchase  by  the  Air  Corps  of  desiqns  completed  by  competing  contractors. 
The  Air  Corps  in  the  1930s  went  beyond  the  concept  of  design  competition 
and  began  asking  competing  contractors  to  submit  a  full-scale  flying 
prototype  of  their  design.  The  prototype  was  requested  to  correct  the 
problem  of  unproven  technical  ideas  beinq  submitted  during  paper  design 
competition  and  subsequently  developed  aircraft  being  much  different 
from  submitted  desiqns. 

The  technical  complexity  of  systems  continued  to  increase  during  the 
1930s  to  the  point  where  the  costs  of  prototype  development  and  the 
exigencies  of  World  War  II  discouraged  the  development  of  prototypes. 
For  example,  Merton  Peck  and  Frederic  Scherer,  writing  at  the  Harvard 
University,  estimate  the  prototypes  in  the  early  years  of  military 
aviation  cost  as  little  as  $6,000  to  $10,000.  By  1939,  however,  this 
cost  had  risen  to  $600,000  and  by  the  lP40s  averaged  several  million 
dol  1  ars  each.  H 

The  costs  of  prototype  development  increased  after  World  War  II. 
For  example,  initial  jet  fighter  nroto types  in  the  1950s  cost  $10  to  20 
million,  a  cost  that  the  Air  Force  felt  was  too  much  to  spend  on 
prototyping.  Consequently,  the  emphasis  shifted  from  prototype 

competition  back  to  design  competition. 

Technology  continued  to  develop  at  an  increasingly  faster  pace 
after  the  Korean  War,  as  did  the  perceived  military  threat  to  the 
nation.  The  United  States  capitalized  on  state-of-the-art  hi  qh 
technology  to  counter  the  threat.  Feasibility  studies  of  the  basic 
underlying  scientifir  concepts  preceded  the  design  of  the  system. 
Management  competition  was  the  answer  to  these  new  technological  deve¬ 
lopments  as  the  cost,  of  parallel  systems  des  i  qn  became  increasingly 
expensive.^  Under  management  competition,  the  contractor  submits  the 
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general  technical  and  organizational  approach  he  will  use  in  e,p  :v 
the  feasibility  of  a  given  technology.  The  best  submission  d  j - ; ;  the 
competition  is  chosen  as  the  development  contractor. 

Coupled  with  the  galloping  technics  1  advance  which  occurred  g., >  eg 
this  period,  the  launching  of  Sputnik;  by  the  Russians  in  1%/  create  j  a 
sense  of  urgency  to  decrease  the  developmental  time  associated  wit" 
major  weapon  systems.  One  congressional  report  referred  to  this 
by  saying: 


If  the  Pearl  Harbor  attack  unified  this  nation  an' 
mobilized  its  people  for  World  War  II,  Sputnik  was  -n 
electrifying  event  of  similar  impact  for  a  cold  war  decade 
missile  and  space  development.  it  served  to  mobilize  *.!S 
science  and  technology  through  the  creation  of  new  agencies 
and  offices,  subsidies  t.o  higher  education,  space  ventures 
which  led  finally  to  a  triumphant  landing  on  the  moon,  and 


defense  efforts  which  greatly  expanded  strategic  forces  'rO' 
deterrence  against  nuclear  war.^j 


Procurements  during  this  period  generated  the  large  cost  overruns 
of  the  1960s  and  led  to  a  general  concern  by  the  Congress  regari,nq  the 
acquisition  process.  Dr.  John  Foster.  Director  of  Defence  Pesejrch  and 
Engineering,  in  testifying  before  f.he  Congress,  stated  how  the  exisr.irg 
national  situation  and  a  concern  for  weapon  system  development  time 
created  the  problems  of  the  1960s. 


Certainly  we  came  out  of  the  fifties  with  a  practice  that 
in  fact  arose  from  the  national  situation.  You  will  recall  we 
were  faced  with  an  alleged  "missile  gap."  We  thought  we  had  a 
strategic  problem  on  our  hands  and  were?  forced  to  develop 
weapon  systems  that  had  not  been  thoroughly  engineered  and 
risks  removed.  These  were  not  representative  of  the  state  of 
the  art.  The  art  had  to  be  put  together  and  produced.  There 
was  at  that  time,  a  great  deal  of  development-production 
concurrency.  It  was  based  on  a  felt  need,  and  policies  had  to 
be  flexible  enough  to  meet  that  kind  of  practice.^ 


Inis  concern  for  lead  t.  i  mo  resulted  in  several  acquisition  5 /Stem 
changes.  Time  schedules  were  compressed  through  the  use  of  concurrency. 
Production  overlapped  development  and  deployment  overlapped  bafn 
development  and  production.  "Crash  programs  were  instituted.  Spec" a 
offices  were  created  with  authority  to  command  the  needed  resources  for 


system  development, 
channels  with  higher 


and  to  communicate 
ec.he  1  ons  .  •’ 


quickly  through  stream! i new 


study  referred  t  a  0  a  ••  1  i  , 
e  t no  act  ii  I  r n  *  1 


Pock  and  Scherer,  i  n  their  definitive 
found  that  during  this  period  of  tne  "missil 


weapons  systems  varied  from  original  estimates  by  a  factor  of  3.2; 
developmental  time  was  L.36  times  longer  than  forecast;  and  the 
performance  levels  of  developed  systems  ranged  from  .'3  to  2  times 
projected  levels.^  They  attributed  these  results  to  technological 
advances  which  created  uncertainty  at  the  outset  of  the  program,  the 
sense  of  urgency  which  existed  during  the  missile  race,  and  the 
concomitant  lack  of  regard  for  program  cost. 

In  1961  Robert  S.  McNamara  became  secretary  of  defense.  The 

perceived  "missile  gap"  was  found  nonexistent,  and  he  attempted  to  place 
as  much  importance  on  cost  as  the  time  factor  in  the  acquisition  of 

weapon  systems.  One  of  his  innovations  was  total  package  procurement 
!TPP),  a  contracting  method  in  which  the  contractor  agreed  to  develop  a 
weapon  system  which  met  required  performance  specifications  at  a  given 
cost.  According  to  McNamara,  TPP  would  broaden  competition  and  reduce 
the  use  of  cost-plus  f i xed-fee-contracts . ^  It  also  shifted  major  cost 
risks  to  the  contractor  for  developing  state-of-the-art  technology. 

The  C-5  aircraft  was  the  first  major  program  to  be  procured  using 
the  total  package  procurement  acquisition  method.  Contracts  were 
written  early  in  its  infancy  on  a  f i  nn-f i xed-price  basis.  However, 
because  cost  was  locked  in  too  early  in  the  procurement  process, 

Lockheed  almost  went  bankrupt  and  the  structural  integrity  of  the  C-5 

winy  is  such  that  it  is  currently  undergoing  a  major  winy  modi f i cat i on . 
Total  package  procurement  as  a  concept  died  after  this  experience  and 
the  use  of  cost-type  contracts  again  increased  with  technical 
competition  being  the  primary  selection  criterion. 


Type s  of  C ojnp e t  i  t  i  on 


This  rather  terse  review  of  the  evolution  of  the  systems 

acquisition  process  reveals  that  as  systems  became  more  technically 
complex  and  expensive,  the  selection  of  system  contractor  was  made  on 
technical  considerations  early  in  the  acquisition  process  to  minimize 

the  expenditure  of  time  and  development  cost.  Stated  differently, 
program  managers  selected  the  winning  contractor  as  early  in  the  process 
as  possible  r o  minimize  the  costs  of  keeping  two  contractors  involved 
through  ESE1). 

These  trade-offs  between  time  and  cost  are  substantial  because  the 
level  of  research,  development,  test,  and  evaluation  {ROTSE) 
expenditures  on  a  major  program  is  high,  lor  example,  as  shown  in  Table 
1,  Column  4,  the  estimated  base-year  R9I  SE  costs  for  the  4-19  at.  source 
selection  were  budgeted  at  ?91.9  million  in  TV  H/9  dollars.  The  3-lR 
program  baseline  RIHSE  costs  were  estimated  at  53  billion  in  I- Y  199,1 
dollars.  Competition  strategy  for  the  4-19  involved  a  prototype 

competition  between  the  Northrop  4-9  and  Fairchild  4-19  with  the  source 
selection  being  made  at  the  beginning  of  I  Si  > .  If  competition  bad  been 
continued  into  FNi  1  for  the  4-19  ,  >  r  the  of  her  »>r  lgrams ,  99  f  costs 

wool  t  have  been  even  higher.  It  or'  ma  ;  v  weapon  sv,t ems  programs  as 


shown  in  Table  1-1  have  used  a  similar  strategy  So  select  the 
sole-source  contractor  at  initiation  of  FSEd  to  minimize  up-front 
development  costs. 

While  competition  was  used  to  select  the  winnnin<j  contrast  or  in 
these  programs,  a  distinction  mist  lie  made  between  price  and  technical 
competition.  As  is  shown  in  Column  7,  rtDT&E  expenditures  is  ; 
proportion  of  total  acquisition  costs  range  from  a  low  of  0  po -  _:r 
the  F -16  to  39  percent  for  the  E-3A  aircraft  in  base-year  dollars.  :-r  : 
considering  that  total  ROI&F  expenditures  include  full-scale  engineering 
development  costs,  source  selection  was  made  in  the  F - 1 6  prog'-v-  witn 
less  than  h  percent  of  program  expenditures  made.^  Examiner  : -  •  -  e ' 
way,  competitive  price  pressures  for  the  production  stage  '  the 

acquisition  were  removed  based  on  technical  competition  with  less  man  0 
percent  of  program  expenditures  incurred.  Similar  inferences  "an  np 
made  with  data  for  other  programs  listed  >0  the  ladle.  And  because  of 
the  technical  and  cost  uncertainties  remaining  i  n  the  program,  tr.-n.  ; 
an  inability  to  lock  in  contras tua lly  at  the  pre-FSED  stage  the  benefits 
of  competition  for  pricing  of  weapons  in  production. 

Exploring  the  distinction  between  technical  and  price  competition 
more  closely,  figure  1-3  relates  types  of  competition  with  the  stage  of 
the  acquisition  process.  Early  in  the  process,  oaper,  management,  and 
prototype  technical  competition  prevails.  And  because  of  toe  teconica1 
uncertainty,  production  costs  cannot  be  estimated  with  any  degreu  of 
confidence  and  contractors  are  unwilling  to  enter  into  f  i  xed-pri. co¬ 
product  ion  contractual  arrangements.  Accordingly,  the  government  ises 
cost-type  contracts  and  bears  the  risk  of  cost  uncertainty.  As  system 
characteristics  and  baselines  became  timer,  use  of  fixed-price 
contracts  and  price  competition  increases. 

The  Office  of  Federal  Procurement  Policy  states  that  "competition 
can  involve  evaluation  of  new  ideas,  productivity  improvements,  new 
products  and  cost  factors,  as  well  as  a  firm's  technical  and  management 
ability."^  Paper,  management  in-l  prototype  competitions  have  a’ready 
been  discussed  and  it  was  spawn  that  source  selection  using  toose 
methods  was  based  on  technical  competition.  A  brief  description  of 
the  types  of  price  competition  is  given  to  illustrate  their  use  to 
stimulate  price  competition,  p’ther  singularly  or  in  combination,  dur'ng 
the  production  stage,  while  eliminating  some  of  the  duplicate  up-front 
F3EI1  costs.20 


technical  data  Package  ( TOP ) 

The  TUP  is  the  technical  description  of  an  item  adequate  for  ,  ,  .•  in 
procurement.  This  description  defines  the  required  design  conf  1  gu  -a  t  i 
and  assures  adequacy  of  item  performance.  It  r-'ovidrs  the  nas-’l  ’re 
against  which  contractors  ran  suhmir  f 1 r  n- f i xe  f-pr ;  ru*  hius. 
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CONCEPT  EXPLORATION'  DEMONSTRATION  FULL  SCALE  PRODUCTION 
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Fors n F_i  t_,__a_n_ d  Junction  ( F  ^ ) 

The  form,  fit,  and  function  (F-M  method  is  the  description  of 
military  equipment  by  performance  character] st i cs .  The  equipment  is 
described  in  terms  of  output,  function,  and  operation.  External 
configuration,  mounting  provisions,  or  interface  requirements  may  be 
included.  Rut  details  of  design,  fabrication,  and  internal  structure 
are  normally  left  to  the  option  of  the  contractor. 


Lea d_e r  / J o  1  _1  ow e r  ( L  / F  )_ 

The  leader/follower  (L/F)  method  is  an  acquisition  technique  under 
which  the  developer  or  sole  source  producer  of  an  item  or  system  (the 
leader  company)  furnishes  manufacturing  data,  assistance,  and  know-how, 
or  otherwise  enables  the  follower  company  to  become  a  source  of  supply 
for  the  item  or  system. 


Jducatj  on  a  1_  B_uy 

An  educational  buy  is  a  contract  to  provide  a  firm  the  opportunity 
to  learn  how  to  manufacture  limited  production  quantities  of  a  military 
item  of  equipment  in  accordance  with  a  government  TOP.  Normally,  the 
purpose  of  the  method  is  to  generate  a  competitive  second  source  for  an 
item  which  has  previously  been  bought  noncompeti ti vely . 


Oi_r e ct ed  Licens in g_ 

The  directed  licensing  method  is  akin  to  leader/follower  in  that 
the  leader  provides  data  and  assistance  to  help  a  follower  become  a 
qualified  producer.  However,  with  licensing  not  only  is  assistance 
provided  but  the  developer  (who  may  be  the  leader  or  subcontractor  of 
the  leader)  is  selling  or  renting  something  he  owns  (patents,  trade 
secrets,  etc.). 


Contractor  _Team  Arrangements 

Contractor  team  arrangements  involve  a  prime  contractor  arrangement 
where  two  or  more  companies  form  a  partnership  or  joint  venture  to 
compete  as  a  potential  prime  contractor  using  their  combined  knowledge 
and  abilities. 


Cop  r_o duct  i  on 

Under  this  concept  two  or  more  prime  contractors  develop  or  produce 
different  subsystems  or  major  components  of  a  weapon  system.  They  then 
transfer  learning  to  the  other  contractor  and  both  compete  against  each 
other  for  contract  awards. 


IP 


Component  Breakout 

Component  breakout  is  the  process  of  d i v i  ling  an  end  item  in"  i  - 
component  parts  so  that  the  component"'  may  be  purchaser!  di^ecM,- 
manufacturer  rather  than  f rom  the  end  item  prime  contractu--.  ^ 

"component1  includes  subsystems,  assemblies,  subassemblies,  ana  rei)1v- 
parts . 


Competition  Savings 


These  methods  can  he  used  to  stimulate  price  compet  i "  ;  a  eg 
possible  cost  avoidances  during  the  production  of  weapons.  An  ,  .  inple 
should  help  the  reader  visualize  how  this  is  accomplished. 

The  budget  profile  in  the  typical  program  includes  all  nonrecurring 
costs  such  as  development  and  tooling  costs.  It  also  ioc'ides 
recurring  or  variable  production  costs.  A  typical  proqram  total  cost 
profile  reflecting  these  costs  over  time  is  nominally  shown  in  figure  1-4 
and  reflects  early  selection  of  the  prime  contractor  based  on  technical 
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FIGURE  1-4 


competition  with  a  sole-source  production  follow-on  contract.  This 
graph  represents  the  algebraic  summation  of  total  program  expenditures 
for  a  given  program  over  time  as  reflected  in  figure  ?. . 

Assuming  that  price  competition  in  production  is  desired  after  the 
design  of  the  system  is  accomplished  by  the  sole-source  contractor,  a 
second  source  could  be  developed  using  one  of  the  variety  of  strategies 
outlined  earlier.  This  strategy  requires  a  substantial  up-front  initial 
investment  to  enable  the  second  source  to  produce  the  item.  These 
investments  ire  usually  made  with  the  anticipation  that  out-year 
recurring  expenditures  will  be  reduced  enough  to  offset  the  additional 
investment  in  developing  the  second  source.  The  situation  can  be 
depicted  as  shown  in  figure  1-5.  Cn  reflects  the  total  cost  profile  for 
noncompetitive  production  procurement  and  Cc  reflects  the  competitive 
total  cost  profile.  Cc  is  initially  above  Cn  which  reflects  the 
increased  up-front  funding  required  to  develop  the  second  source  as 
shown  by  its  intercept  on  the  Y  axis.  Cc  eventualy  intersects  Cn 
because  the  recurring  costs  in  the  competitive  situation  are  less  than 
in  the  sole  source  situation  as  reflected  in  the  slopes  of  the  total 
cost  curves.  The  area  between  Cn  and  Cc  where  Cn  lies  above  Cc  reflects 
cost  avoidances  to  the  government.  It  is  this  area  that  previous 
studies  have  attempted  to  predict  by  quantifying  the  up-front  investment 
and  the  reduced  recurring  program  costs. 
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FI  CURE  1-5.  Cumulative  Program  Cost 


Del  imits 

Several  previous  studies  have  attempted  to  quantify  the  costs  an d 
benefits  derived  from  dua  I -sourcing  by  examining  program  data  ex  pest 
to  determine  the  impact  of  introducing  competition  into  the  product ’on 
phase.  This  study  is  delimited  to  the  dual -sourci ng  decision  of  weapons 
in  production  for  three  reasons:  first,  because  of  the  high  percentage 
of  procurement  dollars  spent  noncompetiti vely  for  the  production  of 
weapons;  second,  increased  pressure  to  secure  competition  in  production 
is  developing,  and  third,  many  studies  have  been  published  which  use 
different  methodologies  to  arrive  at  different  conclusions  when 
analyzing  the  same  dual-source  procurements. 

We  have  already  alluded  to  the  first  of  these  reasons.  The  initial 
two  stages  of  a  weapons  procurement  in  most  cases  involve  technical 
competition  as  two  or  more  contractors  vie  for  selection  of  their 
technical  concept  by  the  government  on  the  basis  of  its  technical  merit. 
In  contrast  to  this  process,  which  is  coded  as  competitive  negotiation, 
the  majority  of  follow-on  production  contracts  are  awarded 
noncompetiti vely .  Therefore,  to  increase  competitive  awards,  more  must 
be  known  about  the  largest  noncompetitive  award  category  and 
noncompetitive  follow-on  procurements. 


Secondly,  legislative  and  executive  branch  pressures  are  being 
applied  to  increase  production  competition.  Finding  19  from  the  1973 
Project  Ace  study  done  by  Air  Force  Systems  Command  stated  "Second- 
sourcing  to  preserve  competition  and  reduce  cost  is  not  being  utilized 

March  1981,  Deputy 
h  i  s  Acqu i s i t i on 


in  all  appropriate  i  nstances . M<L  ^  More  recently,  on  2 
Secretary  of  Defense  Frank  Carlucci  initiated 
Improvement  Program  by  chartering  five  working 
recommendations  on  improving  the  acquisition  process, 
the  working  groups  provided  their  recommendations 
published  his  31  initiatives  on  30  April  1981.  On 
added  the  32nd  i  ni  t i at i ve--compet i t i on--speci f i ca  1  ly 
to  second  sourcing. 


groups  to  make 
On  31  March  1981 
to  Carlucci  ,  who 
27  July  1981,  he 
referring  in  part 


The  President  issued  Executive  Order  12352  on  17  March  1932 
directing  agencies  to  develop  criteria  to  enhance  competition  and  limit 
noncompetitive  actions.  In  response  to  the  executive  order.  Secretary 
of  Defense  Casper  W.  Weinberger,  in  a  9  September  1982  memorandum 
stated: 


The  Department  of  Defense  components  are  to  place  maximum 
emphasis  on  competitive  procurement.  ...  All  personnel 
involved  in  the  acquisition  process  from  the  first 
identification  of  the  requirement  through  the  execution  of  the 
purchase  should  recognize  this  responsibility. 

President  Reagan  reaffirmed  his  emphasis  to  the  heads  of  departments  and 
agencies  in  an  11  August  1983  memorandum,  which  stated  in  part: 

Numerous  examples  of  waste  and  exorbitant  costs  due  to  fhr 
lack  of  competition  have  been  detailed  by  thp  Congress  and  tne 
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press  during  recent  months.  Although  efforts  have  been 
initiated  by  this  Administration  through  the  Reform  ‘88 
Management  Improvement  Program  to  correct  this  long  standing 
problem,  I  am  convinced  that  more  needs  to  be  done.“ 


He  directed  that  the  Office  of  Federal  Procurement  Policy  issue  a  policy 
restricting  use  of  noncompetitive  procurement  practices. 

The  Congress  has  emphasized  dual-sourcing  in  recent  legislation. 
In  its  DOD  appropriation  legislation  for  FY  1984,  the  Congress  directed 
that  the  Secretary  of  Defense  submit  to  the  Congress  prior  to  the 
initiation  of  full-scale  engineering  development  of  any  major  weapons 
system,  either: 

(a)  a  certification  that  the  system  or  subsystem  being  developed 
will  be  procured  in  quantities  that  are  not  sufficient  to  warrant 
development  of  two  or  more  production  sources,  or 


(b)  a  plan  for  the  development  of  two  or  more  sources  for  the 
production  of  the  system  or  subsystem  being  developed.^ 

Additionally,  legislation  has  been  introduced  into  the  Senate  ( S - 1904 ) 
which  would  require  the  Defense  Department  to  increase  the  amount  of  its 
contract  dollars  awarded  by  competitive  formal  advertising  by  5  percent 
per  year  until  70  percent  of  all  awards  are  competitively  bid.  The 
cumulative  effect  of  these  actions  indicates  the  President  and  the 
Congress  are  serious  about  more  production  competition  through  dual¬ 
sourcing.  Consequently,  the  Department  of  Defense  must  better 
understand  how  to  budget  for  procurements  that  involve  dua 1 -sourcing. 


The  third  reason  this  study  is  delimited  to  dual-sourced  production 
procurements  is  that  many  existing  studies  outline  various  methodologies 
available  to  quantify  the  costs,  benefits,  and  budget  profiles  of  dual- 
sourced  procurements .  When  these  studies  analyze  the  same  dual-sourced 
procurements  using  different  analytic  methods,  they  develop  different 
savings  estimates.  More  important,  however,  is  that  none  of  these 
studies  of  several  dual-sourced  programs  has  identified  a  consistent  set 
of  cost  and  benefit  relationships  existing  between  programs.  Some 
programs  cost  the  government  more  as  a  result  of  dual-sourcing  while 
others  cost  less.  Something  must  be  done  to  clear  up  this  confusion. 


Program  managers  and  budget  personnel  must  better  understand  the 
impact  of  dual -sourcing  on  their  time-phased  budget  so  they  can  balance 
the  increased  emphasis  on  production  competition  with  the  uncertainty  of 
its  results.  This  is  particularly  important  in  light  of  the  fact  that 
budgets  for  each  weapon  system  are  done  by  program  on  a  one-year  basis. 
If  the  estimates  of  the  budget  profile  are  wrong,  overruns  could  occur 
that  would  lead  to  severe  congressional  and  public  criticism. 


This  chapter  presented  the  research  questions,  and  differentiated 
between  the  independent  concepts  of  technical  and  price  competition  and 
how  they  are  used  in  the  different  stages  of  the  acquisition  process. 


It  also  identified  technological  and  cost  uncertainty  and  the  high  cost 
of  developing  weapons  as  the  major  reasons  why  sole-source  awards 
account  for  a  high  percentage  of  the  total  dollar  value  of  contract 
awards.  Moreover,  as  was  also  seen,  the  executive  and  legislative 
branches  are  exerting  pressures  to  increase  price  competition  during 
follow-on  procurements  through  use  of  one  or  more  of  the  techniques 
described.  Finally  it  identified  the  confusion  of  equating  formal 
advertising  with  competition. 

Chapter  2  wi I  1  identify  the  costs  and  benefit  factors  that  must  no 
considered  in  a  dual -sourci ng  decision. 
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CHAP n-.ri  2 


PRICE  COMPETITION:  COST  AND  BENEFIT  FACTORS 

As  stewards  of  the  taxpayer's  dollar,  acquisition  personnel  e  st  n 
concerned  with  the  prudent  expenditure  of  public  funds.  A  decision  to 
dual-source  a  weapon  system  during  production  results  in  add i *  ■  c s I 
costs  and  potential  benefits  to  the  government.  If  the  decision  is  air- 
on  the  basis  of  cost  reduction,  these  costs  and  benefits  msst  r,e 
quantified.  They  could  then  be  compared  to  an  estimate  of  what  tne 
sole-source  price  would  probably  have  been  in  the  absence  of  comp--"  i  t !  on 
to  identify  the  least-cost  method  of  procuring  the  weapon  system.  Or 
if  competition  is  instigated  for  other  reasons,  such  as  enhancing  the 
production  base,  cons i derat  ion  of  the  costs  and  benefits  will  enhance 
the  accuracy  of  the  time-phased  budget  profile  projections.  They  mast 
a  iso  be  used  to  evaluate  studies  that  estimate  the  cost  impact  of 
competition  on  dual -sourced  programs.  This  chapter  presents  a  taxonomy 
of  potential  costs  and  benefits  that,  depending  on  dual -$ourci ng 
techniques  chosen,  might  have  an  impact  on  weapon  system  cost  and  mast 
therefore  be  considered  in  the  dua 1 -sou re i ng  decision. 


Cost  and  Benefit  Elements 

In  his  classic  analysis  of  the  weapons  acquisition  process, 
Frederic  Scherer  of  Harvard  University  pointed  out  potential  compet’t'on 
benefits  when  he  noted  that  recurring  costs  in  the  World  War  I!  bo  ,oer 
program  were  lower  in  competitive  as  compared  to  noncompetitive 
procurements.^  In  contrast,  he  states  that  recurring  costs  for  World 
War  II  fighter  programs  were  higher  in  the  competitive  situation  than 
were  sole-source  procurements.  He  also  concludes  that  competition  is 
not  appropriate  in  all  situations  because  of  potential  costs  of  dual¬ 
sourcing. 


Scherer  identifies  the  potential  cost  in  tnansferring  learning  f t-nn 
the  original  producer,  usually  the  developer,  to  the  second  source. 
Another  cost  occurs  because  of  the  learning  phenomenon,  a  cost  advantage 
accruing  to  the  original  producer.  He  also  states  that  economi es-of- 
scale  can  be  upset  by  splitting  orders  and  also  by  a  company  putting  its 
nest  personnel  on  other  programs.  He  notes  the  costs  of  additional 
tooling  for  second  sources  and  also  that  they  might  change  production 
tolerances  from  those  of  the  original  producer,  thus  (II  complicating 
the  problem  of  field  maintenance  and  (2)  increasing  logistics  support, 
costs . 


Scherer  identifies  many  pertinent  elements  that  must  be  analyzed 
when  considering  dua 1 -sou re i ng  to  obtain  reduced  costs  throuqn  rr  ce 
competition.  He  does  not,  however,  provide  an  exhaustive  catec-vi  zat '  r' 
of  them  or  the  statistical  treatment  required  to  obtain  statistics1!.' 
significant  conclusions,  both  of  which  are  rpceo  -.ary  to  identify  t"e 
total  impact  of  dual -sourcing. 


V  1 


The  impacts  of  dual -sourcing  can  be  identi  :■■■<*  as  those  that  can  be 
quantified  and  those  that  cannot.  Moreover,  ,e  quantifiable  impacts 
can  be  divided  into  nonrecurri ng,  recurring,  other  cost  and  profit. 
Figure  2-1  depicts  each  of  these  categories  and  the  specific  elements 
that  must  be  considered  within  each  category. 

Each  element  will  not  impact  equally  on  all  competitive  dual-source 
procurements.  For  dual -source  competition  to  be  implemented,  two 
criteria  must  be  satisfied.  First,  the  weapon  system  description  must 
be  detailed  enough  for  potential  contractors  to  know  exactly  what  is 
needed  by  the  government.  And  second,  there  must  be  more  than  one 
contractor  capable  and  willing  to  produce  the  weapon  system.  Weapon 
system  complexity  directly  affects  satisfaction  of  both  criteria  and  in 
part  determines  the  exact  impact  of  the  elements  listed  in  figure  2-1  on 
a  specific  program.  Moreover,  the  technique  chosen  to  implement  dual- 
sourcing  will  also  affect  these  costs.  A  detailed  discussion  of  each 
element  will  more  clearly  focus  on  the  variable  impact  each  can  have  on 
different  programs. 


flu  anti  f  i  abl  e  I  mp  act  s 

Non  recu  r_r  i_n_i_C_os_t_s 

The  government  incurs  one-time,  up-front  nonrecurring  costs  to 
develop  the  second-source  capability  to  produce  the  weapon  system 
designed  by  the  developing  contractor.  These  costs  include  selection  of 
the  second  source,  its  subsequent  technical  development,  and  other 
possible  costs. 

Second  Source  Selection.  Source-selection  costs  are  incurred  by 
both  the  government  and  the  contractor.  They  include  the  expense  of 
preparing  and  responding  to  a  request-f or-proposa 1 s ,  evaluating  the 
submitted  proposal  and  accomplishing  pre-award  surveys  of  the  second 
source  to  evaluate  its  ability  to  produce  the  weapon  system. 

The  Air  Force  does  not  systematically  collect  data  on  its  own 
overhead  expense  for  accomplishing  these  tasks.  However,  costs  include 
administrative  salaries,  travel  expenses,  and  miscellaneous  expenses 
such  as  printing  and  facilities  for  both  issuing  the  request-for- 
proposals  and  evaluating  proposals  during  the  source-selection  process. 

The  costs  associated  with  contractor  bidding  and  proposal  expenses 
are  identifiable.  DOD  cost  principles  allow  recovery  of  bidding  and 
proposal  expenses  as  an  overhead  expense.  As  such,  they  are  allocated 
across  all  contracts  in  the  plant  (or  other  profit  center)  in  accordance 
with  company  procedures  specified  in  their  Cost  Accounting  Standards 
Disclosure  Statement,  because  of  the  allocation  process,  some  bid  and 
proposal  costs  will  be  paid  for  on  other  government  contracts  included 
in  the  allocation  base  and  must  be  identified  in  the  dual -sourci  ng 
decision.  More  will  be  said  about  the  impact  of  the  costs  of  one 
program  on  other  government  programs  later  in  this  chapter. 


Quantif i ables 

Nonrecurring  Costs 

•  Second-Source  Selection 

•  Second-Source  Development 

•  Technical  Data  Package 

•  Special  Tooling  and  Test  Equipment 

•  Technical  Transfer 

•  First  Article  Testing 

•  Contingent  Liabilities  for  Undepreciated  Assets 
Recurring  Costs 

•  Cost/Quantity  Relationsh  ips 

•  Production  Rate 

•  Learning  Curve 

•  Contract  Administration  Costs 

•  Technical  Data  Update 

Other  Costs 

•  Company  Funded  R&D 

•  Spare  Parts 

•  Overhead  Impact 

•  Logistics 

Profit 


Nonquantif ia b le s 

•  Product  Quality 

•  Cooperation 

•  Technical  Transfer 

•  On-Going 

•  Time  Delays 

•  Claims  and  Performance  Failures 


Figure  ?-l.  Cost  and  Benefit  Impacts  of  Dual -Sourr inq 


Second  Source  Development.  The  financial  commitment  by  the 
government  to  develop  the  second  source  usually  involves  the  preparation 
and  validation  of  the  technical  data  package  and  its  transfer  to  the 
second  source.  It  can  also  include  the  cost  of  special  tooling,  test 
equipment,  and  first  article  testing. 

The  government  pays  for  weapon  system  and  manufacturing  design 
during  full-scale  engineering  development  and,  except  for  products  or 
processes  developed  at  contractor  expense,  owns  rights  to  the  technical 
data  that  it  has  paid  to  develop.  However,  the  design  and  process  data 
are  usually  dispersed  throughout  the  contractor  and  subcontractor 
facilities  and  are  not  readily  accessible  for  transfer  to  a  second 
source.  The  government  must  pay  for  collecting  and  reproducing  this 
data  which  in  some  cases  can  include  2 bO ,000  to  300,000  parts  involving 
20,000  to  30,000  drawings  and  an  unknown  number  of  processes. 

A  related  data  issue  involves  designs  or  processes  developed  by  the 
contractor  at  its  own  expense.  Proprietary  information  is  the  property 
of  the  developer  and  must  be  licensed  or  purchased  by  the  government  for 
use  by  another  contractor. 

The  technical  data  package  must  be  validated  to  assure  its 
acceptability  for  production  use.  Either  production  by  the  sole-source 
contractor  or  examination  by  an  external  activity  can  accomplish  this. 
For  example,  the  Naval  Avionics  Center  validated  the  data  used  for 
second-sourcing  of  the  Phase  III  user  ground  equipment  for  the  Navstar 
3 1 oba 1  Positioning  Satellite  System. 

The  technical  data  package  for  less  complex  weapons  might  be  suf¬ 
ficient  for  potential  contractors  to  submit  bids  on  the  required  system. 
However,  in  more  complex  systems,  the  potential  second  source  might 
require  technical  assistance  from  the  system  developer  using  the  leader- 
follower  technique  discussed  earlier.  This  cost  of  technical  transfer 
would  also  be  accompanied  by  additional  costs  to  the  government  for 
monitoring  the  transfer  process. 

There  must  be  a  quantification  of  the  costs  involved  in  the  second- 
source  developing  its  engineering  and  qualification  models  to 
demonstrate  adequacy  of  the  technical  package  and  its  ability  to  pro¬ 
duce.  Also,  any  contract  costs  incurred  to  increase  second-source  manu¬ 
facturing  efficiency  through  education  buys  must  be  considered  to  the 
extent  their  product  cost  is  higher  than  if  the  product  had  been  pro¬ 
duced  by  the  original  source.  Such  cost  differences  could  originate 
because  of  the  economies  of  scale  of  the  original  producer.  More  will 
be  said  on  this  point  later  in  this  chapter. 


Special  tooling  and  test  equipment 
such  a  specialized  nature  ’*■ 

than  that  for  which  it 
or  altered.  And  where  adequate 
government  typically  pays  its  f 
competition  in  follow-on 


is  tooling  and  test  equipment  of 
that  it  cannot  be  used  for  any  purpose  other 
was  purchased  unless  it  is  substantially  modified 
price  competition  is  lacking,  the 
jl!  cost.^  Therefore,  to  stimulate 
procurements,  the  government  generally  must 
p r ) v i  ie  some  to  the  proposed  second  source. 


The  quantity  of  government-furnished  equipment  required  depends  on 
the  size  and  complexity  of  the  program,  the  nature  of  the  equipment,  and 
the  potential  second-source  contractor's  equipment  inventory.  In  most 

cases,  there  is  no  requirement  for  an  exact  duplication  o‘~  fne 

sole-source  contractor's  equipment  lists  since  the  second-source  mi  got 
own  some  of  the  required  equipment.  And,  as  in  the  case  of  education 
buys,  these  costs  should  not  be  considered  to  the  extent  they  would  be 
incurred  to  tool-up  the  original  source  to  expand  its  production 
capacity  in  the  absence  of  dual -sourcing. 

Contingent _ LJabilit ies.  The  government  might  be  contractual ly 

obligated  for  other  costs  to  the  original  producer.  For  ex3;,''ie,  to 

encourage  the  developer  to  invest  in  efficient  equipment,  the  government 
might  have  guaranteed  reimbursement  to  the  contractor  if  the  contractor 
was  not  able  to  depreciate  the  investment  over  a  specified  production 
quantity.  The  existence  of  these  contingent-liabilities  for 

undepreciated  assets  on  existing  contracts  must  be  considered. 


Recur r i_nc[  Co st s 

Recurring  costs  are  variable  costs  that  occur  during  product 1  on  and 
are  dependent  on  the  levels  of  production.  Some  are  directly  associated 
with  the  quantity  produced.  Others  relate  to  contract  admi ni st rat i on 
and  the  costs  for  updating  the  technical  data  package. 

Cost-Quant.  1 1"  Relat don  ships .  Costs  of  production  can  vary  with 
both  the  rat'  r  production  and  tno  maturity  of  the  production  process 
as  measured  by  the  cumulative  quantity  produced.  The  firm's  production 
function  specifies  the  production  rate  impact,  and  its  learning  curve 
measure j  the  cost  impact  of  a  maturing  production  process. 

Production  Rate.  The  economic  theory  of  production  provides  a 
theoretical  basis  for  studying  the  impact  of  production  rate.  The 
production  function  specifies  the  relation  between  the  quantity  of 
inputs,  or  factors  of  production,  and  the  product  or  output  trey 
produce.  Each  firm  has  a  unique  production  function  dictated  by  its 
factors  of  production  and  the  technology  it  uses  to  transform  its  inputs 
into  outputs.  Hence  if  the  efficiency  of  technology  improves,  more 
output  should  be  generated  from  the  same  level  of  inputs.  Factors  of 
production  include  research  and  manufacturing  equipment  and  facilities, 
research  and  manufacturi ng  labor,  and  management. 

Exploring  this  concept  further,  the  factors  of  production  can  ne 
both  fixed  and  variable.  In  the  economist's  concept  of  short-run,  at 
least  one  of  the  factors  must  be  fixed  and  all  others  are  free  to  va^v. 
In  the  long-run,  all  factors  are  free  to  vary. 

For  purposes  of  example,  assume  a  short-run  scenario  where  laud, 
equipment,  and  buildings  are  fixed  and  labor  is  a  variable  input.  As 
output  is  expanded  through  additional  labor  inputs  to  the  fixed  plant 
and  equipment,  hence  varying  *  he  proportions  between  these  inputs, 
output  is  affected  as  shown  by  'he  total  output  line  TP  in  figure  2-2. ^ 


Total  output  increases  up  to  point  C.  However,  the  rate  of  increase 
decreases  between  point  B  and  C  as  measured  by  the  slope  of  TP  which 
represents  the  marqinal  return  or  additional  product  produced  by 
varying  the  proportion  of  one  input. 


<'1  I  .ihnj 


figure  ?-9.  V ari abl e  P roporti ons 


The  concept  of  diminishing  returns  is  more  clearly  seen  in  the 

lower  half  of  fiqure  2-2  by  reference  to  the  average  product  CAP)  and 
marginal  product  (MP )  curves,  both  MP  and  AP  increase  at  first  because 
more  labor  increases  efficiency.  Then  output  declines  as  people  hoqin 
to  get  in  each  others  way  or  there  is  nothing  for  than  to  do. 

T heoret i cal  1 v,  at  point  D,  marginal  product  actual  1 y  decreases. 

The  marainal  product,  of  any  quantity  of  a  variable  input,  depends  on 

the  state  of  technology  and  on  the  amounts  and  quality  of  the  fixed 

inputs.  If  one  firm  had  more  efficient  fixed  inputs  such  as  hi qh- tech 
manufacturing  equipment,  marginal  product  would  he  higher  than  a  firm 
with  older,  less  efficient  equipment.  rvon  with  such  differences,  the 
""essagp  is  clear;  namely,  after  some  po’nt  in  either  case,  total  product 
would  grow  at  a  slower  rate  and  marginal  produrt.  would  diminish. 


The  concept  of  marginal  returns  deals  only  with  physical  product. 
Cost  impact  of  diminishing  returns  is  shown  by  looking  at  the  firms  cort 
curves.  Depicted  in  figure  ?-3  is  a  firm's  total  variable  cost  curve 
(TVC)  derived  from  the  production  function.^  The  cost  arid  product  '<■ n 
functions  are  duals  of  each  other.  That  is,  the  cost  function  can  he 
derived  from  the  production  function  and  vice  versa. 

Total  variable  cost  rises  first  at  a  decreasing  rate  indicating  the 
higher  marginal  product  of  the  inputs.  It  then  rises  at  an  increasioo 
rate,  indicating  the  decreasing  marginal  product.  The  slope  of  TVr  is 
its  marginal  cost  (MC). 

These  cost  curves  reflect  only  variable  costs.  The  average  cost 
curves  (AC)  must  also  consider  average  fixed  cost  (AFC)  as  depicted  in 
figure  2-4.5  /\s  output  is  increased  in  the  short-run,  the  average  cost 

curve  of  a  firm  always  declines  to  a  minimum  and  then  increases.  The 
magnitude  of  decline  depends  on  the  proportion  of  fixed  to  total  costs. 
If  the  proportion  of  fixed  to  total  costs  is  high,  the  decline  in 
average  costs  is  rapid.  And  to  the  extent  that  firms  have  different 
production  technologies  and  accounting  systems,  cost  changes  will  be 
different  between  different  firms. 

The  minimum  point  on  the  average  cost  curve  represents  the  capacity 
at  which  the  plant  was  designed  to  produce  efficiently.  Production 
at  a  rate  either  higher  or  lower  than  the  minimum  is  accomplished  at  a 
higher  average  cost.  How  much  higher  is  reflected  by  the  sione  of  the 
curve  which,  as  already  stated,  is  unique  to  each  firm. 
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Figure  2-3.  Cost  and  Production  Functions 
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Studies  of  the  acquisition  process  have  identified  the  cost  impacts 
of  producti on- rate  changes.  The  Defense  Science  Board  in  its  1977 
Summer  Study  stated  that  very  low  rates  of  production  substantially 
increase  labor  costs  of  airframe  construction  and  assembly,  and  fiqure 
7-5  graph  i  cal  1  y  s  hows  this  relationship.  As  illustrated  a  hypothetical 


Fiqure  ?-4.  Short-Run  Cost  Curves 


Figure  ?-5.  F  f  f  ect  of  Production  Rate  on  Commercial 
Airplane  Airframe  Cost 
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200,000-pound  airframe  produced  at  one  unit  per  month  has  labor  cost', 
about  40  percent  higher  than  the  same  airframe  at  two  units  per  month. 


The  Affordable  Acquisition  Approach  Study  published  by  Ai*-  -orce 
Systems  Command  in  February  1983  studied  109  Air  Force  Systems  and  6  JO 
reports  on  the  acquisition  process.  One  of  the  study's  principle 
findings  was  that  there  had  been  a  significant  decrease  in  annual  weapon 
system  production  rates.  It  estimated  that  a  30  percent  reduction  of 
production  for  a  hypothetical  budget  program  for  FY  83-88  could  create 
an  increase  in  cost  of  from  6  to  14  billion  dollars. ^ 


Former  Deputy  Secretary  of  Defense  Frank  Carlucci  initiated  his 
1983  Acquisition  Improvement  Program  previously  referred  to  in 
Chapter  1.  Initiative  7  related  to  procuring  economic  production  rates 
to  spread  fixed  costs  over  a  relatively  large  number  of  end  items. 
Carlucci  testified  before  the  House  Armed  Services  Committee  that  $2.3 
billion  could  be  saved  by  accelerating  production  rates  on  18  programs 
as  a  result  of  this  initiative.®  Some  of  the  programs  included  the  Navy 
F /A  18  Hornet,  Army's  TOW  anti-tank  missile,  KC-135  reengining,  and  the 
Mark  48  torpedo.  This  estimate  of  savings  was  subsequently  increased  by 
$86.2  mi  1  lion. ^ 


The  Army  Procurement  Research  Office  in  a  1980  study  reviewed 
research  on  the  rate  of  production  over  the  last  30  years  and  its  impact 
on  weapon  system  cost.^  They  conclude  from  their  review  of  previous 
studies  and  their  own  case  analyses  that  the  most  important  contributor 
to  increased  unit  cost  under  rate  changes  is  an  increased  overhead 
al location. 


In  summary  the  economic  theory  of  production  and  published  studies 
both  point  to  a  conclusion  that  production  rate  changes  directly  affect 
the  costs  of  producing  a  weapon  system.  Therefore,  in  estimating  the 
cost  impact  of  competition,  the  effects  of  production  rate  changes  must 
be  separately  accounted  for. 

Learning  Curves.  Learning  curves  are  related  to  total  production 
and  represent  a  separate  cost/quantity  relationship  that  must  be 
examined.  This  concept  was  first  written  about  by  T.  P.  Wright  in  1936 
when  he  related  learning  curves  to  the  aircraft  industry.**-  The  concept 
tends  to  show  that  as  total  cumulative  production  doubles,  labor  hours 
required  to  produce  an  incremental  unit  are  reduced  by  a  constant 
percentage,  such  as  80  percent.  This  reduction  occurs  as  the 
organization  acquires  a  greater  familiarity  with  required  tasks,  becomes 
more  efficient  with  tools  and  procedures,  and  improves  internal 
coordination  until  it  reaches  a  plateau.  Once  this  plateau  is  reached, 
only  production  rate  effects  will  change  the  product  and  cost  functions 
previously  discussed. 

Wright's  original  relationship  was  a  power  function  of  the  form: 


Yb 


a  .b*x 


Where 


> 


b  --  the  total  production 
Yk  --  labor  hours  required  for  the  b^h  unit 
a  --  labor  hours  required  for  first  unit  (is  a  constant) 
x  --  measure  of  the  rate  of  learning 

Transforming  this  model  into  logarithms  you  get: 
log  Yb  =  (log  a)  -x  (log  b) 

This  formulation  is  the  equation  of  a  straight  line  with  slope 
(-x).  Using  the  transformed  data,  the  logarithm  can  be  plotted  on 
standard  graph  paper,  or  the  untransformed  data  can  be  plotted  on 
logarithmic  coordinate  paper. 


The  learning  curve  can  be  developed  using  cumulative  average  hours 
per  unit  or  hours  per  unit  as  the  Y  axis  and  quantity  produced  as  the  x 
axis.  Neither  model  has  been  empirically  validated  as  being  more 
advantageous.  Consequently,  choice  of  either  model  is  at  the  discretion 
of  the  analyst.  However,  several  authors  have  noted  that  models  using 
the  cumulative  average  formulation  can  be  misleading  because  of  the 
power  of  the  averaging  process  to  smooth  the  data  and  enhance  the 
appearance  of  the  curve.  This  is  particularly  significant  when  there 
is  only  a  small  number  of  data  points. 


Other  formulations  of  learning  curves  have  been  postulated.  These 
have  been  set  out  elsewhere  in  the  literature  and  are  considered  beyond 
the  scope  of  this  research.13  However,  we  need  to  look  at  three  points 
regarding  learning  curves. 


First,  it  is  better  to  use  labor  hours  in  the  formulation  of 
learning  curves  instead  of  labor  dollars.  Calculation  of  learning  curves 
using  labor  dollars  includes  the  effect  of  changes  in  wage  rates  and 
benefits  over  time.  Unless  adjustments  are  made  to  the  data  that 
exactly  reflect  these  changes,  the  true  effect  of  "learning"  could  be 
overshadowed.  Using  labor  hours  eliminates  this  potential  problem. 

A  second  point  relates  to  the  aggregation  of  the  learning  curves 
for  the  individual  components  of  product  cost:  for  example, 

manufactured  parts,  various  subassemblies,  purchased  parts,  and  final 
product  assembly.  Each  component  has  its  own  associated  learning 
related  to  the  opportunity  for  learning  discussed  earlier.  Similar 
processes  should  be  aggregated  in  order  to  increase  the  accuracy  and 
reliability  of  estimates  and  to  facilitate  adjustments  for  changes  such 
as  production  technology  or  product  complexity  over  time. 

A  third  point  is  that  there  is  no  agreement  on  a  "fundamental  law" 
of  learning  such  as  the  existence  of  an  80-percent  curve.  Conway  and 
Schultz  conclude 


There  are  significant  differences  in  patterns  of  progress  for 
different  industries,  different  firms,  different  products  and 
different  types  of  work.  ...  No  particular  slope  is 
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universal,  and  probably  there  is  not  even  a  common  model.  The 
contention  that  such  exists  is  most  difficult  to  defend  either 
logically  or  empirically.^ 

Others  have  concluded,  after  empirical  study,  that  "standard  rates  of 
[learning  curve]  progress  can  be  safely  applied  only  at  the  company  or 
facility  level  until  further  research  finds  otherwise."^  Moreover, 
differences  in  accounting  can  lead  to  different  results  between 
companies. 

The  important  point  of  this  discussion  of  the  learning  curve 
phenomenon  for  this  research  is  that  it  exists.  That  is,  over  time  a 
firm  can  reduce  its  cost  of  production  in  a  potentially  predictable 
manner.  However,  any  reduction  must  be  examined  on  an  irdividual  oasis. 

Contract  _A_d_mi_n_i _st_r at .  i_o _n_  Cost s.  DAR  1-406  lists  72  functions  to  be 
performed  by  in-plant  government  personnel.  Some  of  these  functions  are 
independent  of  specific  contracts  and  relate  to  approval  of  contractor 
systems.  However,  some  functions  are  contract-specific  such  as 
production  surveillance,  property  administration,  and  quality  control. 
Contract  administrators  are  responsible  for  monitoring  the  additional 
contract  provisions.  Quality  personnel  must  reject  nonconforming 
material.  And  property  personnel  must  monitor  contractor  accountability 
and  maintenance  of  government  property.  Performance  of  these  contract- 
specific  functions  at  a  second  source  can  increase  government  personnel 
and  associated  costs. 

Technical  Data  Update.  A  configuration  control  process  must  be 
established  to  update  technical  data  packages  used  by  the  second  source 
as  engineering  changes  are  made  during  production  by  the  developing 
contractor.  The  Air  Force  Management  Analysis  Group,  directed  by  Maj 
Gen  Dewey  Lowe,  published  its  report  in  October  1983.  It  concluded 
that  one  of  the  major  reasons  for  the  lack  of  competition  was  tie 
existence  of  outdated  data  caused  by  a  lack  of  control  of  engineering 
changes. Configuration  management  seeks  to  assure  the  currency  of 
technical  information  held  by  the  second  source  and  its  costs  must  be 
considered  in  the  dual-sourcing  decision. 


Other  Costs 

The  potential  cost  and  benefit  impacts  discussed  so  far  nave 
focused  on  impacts  experienced  during  the  production  stage  of  the 
specific  program  in  question.  Costs  can  also  accrue  to  the  specific 
program  in  both  its  research  and  development,  and  its  deployment  phases. 
And  dual -sourcing  can  have  an  impact  on  the  cost  of  other  programs. 

Research  and  Development  Costs.  As  discussed  in  chapter  1.  the 
research  and  development  process  is* 'so  costly  that  contractor  selection 
is  made  early  when  weapon  system  technical  description  and  expected  cost 
are  shrouded  with  uncertainty.  Hence,  source  selection  is  accomplished 


on  perceived  contractor  technical  approach  and  capability  during  early 
development  efforts  when  a  small  percentage  of  total  program  costs  is 
involved.  Moreover,  few  weapons  programs  exist  and  producers  fiercely 
compete  for  these  limited  opportunities. 


Many  studies  of  the  acquisition  process  have  identified  the 
phenomenon  of  "buying-in,"  which  has  its  origins  in  this  technologically 
complex,  cost  uncertain  economic  structure  of  the  defense  industry. ^ 


Buying-in  occurs  when  a  firm  accepts  no  profit  or  incurs  an 
outright  loss  on  the  initial  development  contract  for  a  weapon  system  in 
order  to  put  itself  in  a  sole-source  situation  in  full-scale  engineering 
development  and  production,  where  the  majority  of  costs  are  incurred. 
It  then  expects  any  losses  to  be  made  up  by  contract  changes  made  during 
full-scale  engineering  development  or  production. 


Some  companies  have  stated  that  if  the  opportunity  to  make  up  the 
expenditure  of  their  own  research  and  development  funds  evaporates  in 
the  production  stage  because  of  competition,  they  will  no  longer  incur 
these  expenditures  in  anticipation  of  future  business.  For  example,  in 
a  letter  to  the  Air  Force  regarding  Carlucci ’s  initiative  32  on 
competition,  Goodyear  Aerospace  Corporation  acknowledges  its  buy-in 
strategy  and  notes  its  strategies  will  change  if  competition  is 
introduced.1^  Specifically,  they  state  that  their  research  and 
development  expenditures,  which  are  usually  amortized  over  the  cost  of 
product  hardware,  will  be  reduced  to  fit  a  potentially  smaller 
aftermarket.  Moreover,  they  note  a  similar  spillover  effect  on 
subcontractors  in  that  subcontractors  will  also  recover  all  development 
costs  on  the  initial  buy  instead  of  over  the  life  of  the  program. 
Finally  they  note  a  reluctance  to  stimulate  capital  investment  for  plant 
improvement,  machine  tools,  and  test  equipment.  John  Richardson, 
president  of  Hughes  Aircraft  Corporation,  echoed  these  same  feelings 
about  dual-sourcing  when  he  wrote,  "Some  contractors  will  reduce 
company-funded  research  and  development  expenditures  usually  recoverable 
over  the  cost  of  production  articles. "2° 


The  potential  for  reduced  company-funded  R&D  exists.  If  a  given 
program's  R&D  is  funded  in  part  by  contractors,  and  this  funding  ceases 
as  contractors  perceive  an  inability  to  recover  their  expenditures,  the 
government  is  faced  with  the  requirement  for  additional  up-front  funding 
as  a  result  of  its  dua 1 -sourci ng  decision. 


Spare  Parts.  A  second  area  outside  the  production  stage  that  must 
be  considered  when  evaluating  a  dual-sourcing  decision  is  the  impact  of 
the  availability  of  data  during  weapon  system  deployment  which  would 
allow  for  the  competitive  procurement  of  spare  parts  during  weapon 
system  deployment.  A  June  1983  audit  by  the  General  Accounting  Office 
on  the  D00  high-dollar  spare  parts  break-out  program,  where  parts  are 
bought  from  other  than  the  system  designer,  concluded  that  break-out 
savings  ranged  from  44  to  68  percent.  It  stated,  however,  that 
competition  or  break-out  is  restricted  because  reliable  data  on  actual 
part  manufacturers  are  not  available  for  break-out  purposes  throughout 
the  Ai r  Force. ^1 
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Undersecretary  of  Defense  Richard  De  Lauer  also  noted  that  "one  of 
the  principal  factors  inhibiting  competition  in  procurement  of  spares  is 
the  lack  of  technical  data  with  appropriate  rights  for  procurement 
purposes. Deputy  Secretary  of  Defense  Paul  Thayer  reaffirmed  this 
view  to  the  secretaries  of  the  departments  on  15  March  1983.23 

And  finally  the  Air  Force  Management  Analysis  Group  (AFMAG), 
discussed  earlier,  concluded  that  one  of  the  key  factors  behind  the  low 
competition  rate  was  the  existence  of  inadequate  or  missing  engineering 
data,  for  both  prime  and  subcontractors .24  One  of  the  AFMAG 
recommendations  was  to  develop  an  integrated  spares  acquisition  and 
support  plan  contractual  requirement  during  the  full-scale  engineering 
development  competition.  They  felt  purchase  of  technical  data  at  this 
stage  could  help  harvest  potential  savings  from  the  competitive  purchase 
of  spare  parts. 

Oyerhead_ J_mp_act.*  Indirect  expenses  can  account  for  as  much  as  4U 
to  70  percent  of  contract  expense  and  are  those  expenses  which  cannot  be 
directly  charged  to  a  specific  cost  objective  because  the  costs  of 
accounting  for  them  would  outweigh  any  possible  benefits  to  the 
government  of  charging  them  direct  to  a  contract.  For  purposes  of  cost 
recovery,  indirect  expense  is  distributed  using  some  presumed  general 
relationship  between  it  and  its  benefiting  base.  For  example, 
engineering  building  depreciation  might  be  allocated  to  cost  objectives 
based  on  direct  engineering  hours  or  direct  engineering  dollars.  The 
other  two  commonly  aggregated  overhead  pools  (in  addition  to 
engineering)  are  manufacturing  and  general  and  administrative  expense. 

Another  contract  cost  that  is  allocated  in  a  manner  similar  to 
overhead  expense  is  an  imputed  return  for  facilities  capital  employed  on 
a  contract.  Cost  accounting  standard  414  provides  for  a  return  on 
facilities  employed  in  the  performance  of  government  contracts,  with  the 
facilities  capital  cost  of  money  (FCCM)  allocated  to  overhead  pools. 
FCCM  in-turn  is  allocated  to  benefiting  cost  objectives  along  with  other 
overhead  pools. 

If  the  overhead  allocation  base  for  overhead  pools  is  decreased,  as 
when  business  is  decreased  by  the  split  production  quantities  when 
dual-sourcing,  the  overhead  rate  goes  up  for  all  projects  in  the  profit 
center.  If  these  are  government  programs,  the  net  cost  to  the 
government  will  increase  in  that  facility.  However,  a  decrease  in  the 
overhead  rates  of  the  second-source  should  occur  as  its  allocation  oase 
increases.  Cost  principles  are  inexact  and  costs  and  rates  are 
different  between  production  facilities.  Therefore,  the  net  impact  to 
the  government  on  all  programs  of  splitting  requirements  between  two 
production  facilities  must  be  considered. 

Another  potential  overhead  impact  that  must  be  considered  when 
evaluating  analytical  studies  based  on  cx  post  data  relates  to  a  charge 
in  the  contractor  business  base  as  a  result  of  the  capture  by  the 
contractor  of  other  programs.  This  could  cause  the  overhead  rate  to 
decrease  on  all  programs.  If  these  reduced  rates  were  reflected  in  the 


government's  price  obtained  near  the  time  of  the  introduction  of 
competition,  and  the  pre  and  post  competitive  prices  were  not  adjusted 
to  reflect  this  effect,  the  overhead  reduction  could  i nappropri ately  be 
attributed  to  the  introduction  of  competition.  For  example  the 
economic  production  rate  for  the  F - 1 5  is  now  lower  at 
McOonnel 1 -Dougl as  because  they  are  also  producing  the  F/A  18. 


Npnqu  an  ti  f_i_ab  1  e  1  mp  ac  t  s 

While  not  specifically  quantifiable,  there  are  other  favorable  and 
unfavorable  impacts  on  program  outcome.  J.  C.  McKeown  identifies  some 
of  these  other  factors: 

Discussions  on  competition  frequently  lead  to  concentration  on 
cost  savings  or  price  reduction.  .  .  .  For  DOD,  the  benefits 
of  competition  extend  beyond  just  cost  reduction  to  include 
stimulation  of  innovation  not  only  in  technological  and  design 
areas,  but  also  manufacturing;  lower  unit  costs;  satisfactory 
technical  performance  (and  also  quality);  and  a  strengthened 
industrial  base. 25 

Like  McKeown,  others  have  stated  that  the  evidence  supports  the 
fact  that  quality  and  reliability  of  the  product  are  improved  under 
competitive  conditions.  And  those  interviewed  during  this  research 
stated  that  cooperation  from  the  original  sole-source  contractor 
increased  under  the  threat  of  competition.  They  stated  that  competition 
resulted  in  engineering  change  proposals  being  submitted  faster  and  the 
speed  and  quality  of  other  communications  being  enhanced.  Also,  the 
advanced  fighter  engine  program  office  cited  the  fact  that  in  addition 
to  lower  prices,  product  warranties  were  obtained  during  competition 
which  were  more  favorable  than  the  original  contractor  would  agree  to 
under  sole-source  conditions. 

Three  other  benefits  accrue  to  the  government  under  competition. 
First,  competition  makes  it  possible  to  have  a  surge  capability  at  both 
the  prime  and  subcontractor  level  exists  (as  in  the  case  of  the  advanced 
fighter  engine)  that  can  be  exercised  if  needed  in  the  event  of  a 
national  emergency.  Second,  sources  are  developed  that  potentially 
might  compete  for  future  weapon  systems  requirements.  And  third,  the 
production  base  is  dispersed  which  reduces  the  potential  destruction  of 
the  total  product  production  base. 

Time  delays  associated  with  developing  and  qualifying  the  second 
source  have  a  potential  negative  impact  on  program  completion.  And  risk 
of  failure  for  production  by  the  second-source  exists  as  was  the  case 
for  the  AN/UPM-98  radar  test  sets  and  AN/ARC-31  radio  sets. 26  The  risk 
of  claims  for  inadequate  data  also  exists.  And  finally,  different 
manufacturing  processes  might  lead  to  different  tolerances  or  product 
characteristics  that  could  cause  future  maintenance  or  logistics  support 
problems. 
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The  inability  to  quantify  all  costs  and  benefits  does  not  mean  the 
nonquant i  f  i able  factors  should  be  ignored.  The  preferred  method  wou  1  d  ne  to 
identify  all  costs  and  benefits  subject  to  quantification  and  to 
qual  i  tat  i  ve  ly  describe  the  nonquant  i  fi  abl  es .  Subjective  ju-’p’-nts 
could  then  be  made  by  the  program  manager  as  to  the  cost  effect  ’  vjness 
of  competition  considering  both  the  quanti f iables  and  nonquant i f i an les . 


Statistical  Cons i derat i ons 


Calculation  of  the  net  cost  impact  to  the  government  of  second 
sourcing  involves  recognition  of  three  statistical  and  methods1 ngi cal 
considerations.  First,  contract  data  reflect  then-year  dollars  and 
must  be  adjusted  for  inflation  to  isolate  the  effects  of  competition  from 
those  of  inflation.  Second,  the  flow  of  costs  and  benefits  over  time  is 
not  the  same  for  all  programs.  And,  because  of  the  time  value  of  money, 
these  cash  flows  must  be  put  on  a  com, non  base.  Third,  the  calculation 
of  savings  must  be  done  using  a  common  methodology.  Each  of  these  are 
discussed  below. 


Inflation 

Inflation  is  the  process  of  steadily  rising  prices  which  result  in 
a  decreased  purchasing  power  of  a  given  nominal  sum  of  money.  Demand- 
pull  inflation  occurs  when  aggregate  demand  increases  faster  loan 
output.  Cost-push  inflation  occurs  when  the  prices  of  inputs,  'e.g., 
labor)  increase  faster  than  their  productivity.  In  conducting 
cost-based  time  series  analysis,  the  effects  of  inflation  must  be 
identified  in  the  financial  data  to  more  clearly  examine  the 
relationship  of  the  relevant  variables.  If  the  data  are  not  deflated, 
the  relationship  of  the  impact  of  competition  and  price  could  be  masked 
and  lead  to  erroneous  conclusions. 

Several  indices  are  available  for  use  in  deflating  contract  data. 
Included  are  the  gross  national  product  deflator,  wholesale  price  index, 
consumer  price  index,  or  sub-elements  of  each.  Several  other  indices 
also  exist  that  are  applicable  to  missile  systems.  Whichever  of  the 
indices  is  chosen,  it  must  reflect  actual  or  projected  inflation  on  the 
program  or  subsequent  analyses  will  yield  invalid  results. 


D i scount in g 

As  discussed  earlier,  high-level  executive  and  legislative  prjnqn 
officials  have  strongly  indicated  that  competition  is  good  because  it 
reduces  net  costs  to  the  government,  by  investing  up-front  non necu r r i ^ u 
expenditures  necessary  to  develop  a  viable  second-source,  th-v 
anticipate  that  incremental  cost-,  wall  me  red  iced  by  an  amount.  urcv.-’' 
than  the  up-front  investment.  However,  heca  ise  of  the  tine  vale'  of 
money,  the  problem  i  it  is  1 1  ate  1  y  s  i '-fuses  as  to  the  pattern  of  swings 
over  time  in  relation  t.o  the  up-  *  ••  •n‘.  invest  neot. 


A  simple  illustration  might  clarify  this  concept.  Suppose  you  are 
given  the  choice  of  the  following  investments  and  savings: 


Tear 

Total 

Investment 

Amount 

1 

2  3 

4 

S  a  v i n  gs 

A 

75 

20 

20  20 

20 

80 

B 

75 

80 

80 

C 

75 

80 

80 

All  three  yield  a  savings  of  $80  for  an  investment  of  $75.  However,  all 
the  investments  are  not  equally  attractive.  Investment  B  is  the  most 
attractive  of  the  three  because  the  investment  is  recouped  in  the  first 
year.  This  provides  an  opportunity  to  reinvest  the  money  and  earn 
interest  on  the  savings.  Or  if  the  money  were  borrowed,  the  principal 
can  be  paid  back  to  avoid  interest  charges.  Alternative  C  is  the  least 
attractive  because  for  four  years  interest  that  could  be  earned  by 
investing  the  money  is  foregone. 

Pursuing  this  simplistic  example  further,  hypothesize  that 
the  $75  was  borrowed  at  10  percent  interest.  At  the  end  of  the  year, 
debt  included  the  $75  principal  and  $7.50  interest  for  a  total  of 
$82.50.  A  comparison  of  the  $82.50  cost  to  the  $80  return  dictates  that 
the  prudent  person  search  elsewhere  for  a  more  lucrative  investment. 

In  making  prudent  investment  decisions,  the  present  value  of  the 
entire  cash  flow  must  be  compared  to  the  present  value  of  the  necessary 
investment.  The  formula  for  determining  present  value  of  a  cash-flow 
stream  is: 


P V  =  _R_1  +  _r2  +  f  _Pn 

(1+1T  (TViT2  (1  +  0 n 


where 

PV  =  present  value 
i  =  discount  rate 

Rj,  R2*  .  .  •  Rn  =  cash  flow  in  years  1,  2,  ...  n 
n  =  duration  of  project 

Applying  this  concept  to  the  simplified  example,  the  present  value  of 
investment  A,  assuming  a  discount  rate  of  10  percent,  is: 


V  =  20  +20  +  20  +  20  =  53.42 

(l+.ry  d+.n2  non3  rr+/n4 

cor  investment  1,  the  present  value  is  $72.72,  and  for  investment  C 
the  value  is  $54.79.  Compared  to  the  up-front  investment,  none  of  these 
alternatives  appears  favorable. 
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Federal  policv  requires  the  discounting  of  cash  flows  in  the 
consideration  of  investment  decisions.  0MB  Circular  A-9A  states: 

The  discount  rate  prescribed  in  this  circular  applies  to  the 
evaluation  of  government  decisions  concerning  the  initiation, 
renewal  or  expansion  of  all  proqrarns  or  projects,  other  than 
those  snecifically  exempted  below,  for  which  the  adoption  is 
expected  to  commit  the  government,  to  a  series  of  measunab 1 e 
costs  extending  over  three  or  more  years  which  result  in  a 
series  of  benefits  that  extend  three  or  more  years  beyond  the 
inception  date.?7 

Exempted  from  the  scope  of  the  circular  are  water  resource  project 
decisions,  the  District  of  Columbia  government,  and  nonfederal 
recipients  of  federal  loans  or  grants.  Also  excluded  are  secondary 
decisions  made  to  implement,  a  program  after  a  favorable  decision  to 
initiate,  renew,  or  expand  the  program  is  made  using  the  provisions  of 
the  circular.  In  the  opinion  of  this  researcher,  investments  to  --educe 
the  costs  of  weapons  systems  are  not  exempted  from  this  requirement. 

The  circular  specifies  a  discount  rate  of  10  percent  that  must  be 
applied  to  the  deflated  dollar  flow  of  program  costs  and  benefits  over 
time.  That  is,  the  10  percent  figure  is  to  he  applied  to  inflation- 
adjusted  cash  flows. 

A  henefit-cosf  ratio  is  then  calculated  usinq  the  formula: 

Benefit-Cost.  Ratio  =  PVh*  f  PVC 


whore 


PVb  =  Present,  value  of  benefits 
PVC  =  Present  value  of  costs 

A  ratio  of  greater  than  <mp  would  be  a  favorable  ratio,  with  t^ose 
investments  with  higher  ratios  yielding  the  better  return.  In  our 
simple  example,  the  benefit-rost  ratio  of  alternative  A  is  .79, 
alternative  B  is  .90,  and  alt.ernat.ivo  C  is  .68. 

The  concept  of  discounting  has  become  increasingly  important,  given 
current  economic  conditions.  Present  budget  deficits  are  expected  to  be 
greater  than  $180  billion  a  year  for  the  next  several  years.  Therefore 
the  government  will  have  to  borrow  its  funds  in  the  money  markets  to 
finance  expenditures  at  an  interest  rate  most  likely  greater  than  10 
percent.  Moreover,  16.1  percent  of  the  federal  budget  presently 
consists  of  interest  on  the  public  debt..  Pressures  are  mounting  *  o  cut 
the  debt  and  weapon  system  planners  should  be  sensitive  to  huda-t 
reduction  pressures.  It  escapes  logic  to  borrow  money  at  a  given  rate 
of  interest,  use  it  to  develop  a  second-source  for  the  purpose 
decreasing  weapon  system  cost,  -ind  receive  a  rate  of  return  on 
investment  that  is  loss  than  the  loan  interest  •-ato.  Discount  in  i  as  a 
technique  considers  the  a  1  t^nn.-g ->  investment  opportunities  and  ( act  A»"s 
these  into  the  cost/benefit  ca-.h  f 1  >■<:  stream  over  time. 


Savings  Calculat ion 

The  calculation  of  percentage  savings  attributable  to  competition 
should  include  all  the  variables  discussed  so  far,  including  recurring 
and  nonrecurring  costs.  The  percentage  savings  calculation  could  take 
many  possible  forms,  and  the  reader  is  cautioned  to  understand  what 
percentage  is  being  referred  to  when  a  percentage  is  cited  in  a  study. 
For  example,  table  2-1  lists  savings  attributed  to  the  Shillelagh  missile 
in  one  study. ^8 


fable  2-1 

Diversity  of  Savings  Estimates  on  the  Shillelagh 

22  percent  savings  on  competitive  buy-out,  recurring  costs 
only,  presumably  in  1972  dollars.  Sole-source  learning  curve 
exponent  of  -0.233  reported  in  APR  078  used  to  project 
sole-source  price. 


9  percent  savings  on  all  post-competition  production, 
recurring  costs  only,  in  1972  dollars.  Sole-source  learning 
curve  exponent  of  -0.233  from  APR078. 

-1  percent  loss  on  first  competitive  split-buy,  presumably 
in  1972  dollars,  using  the  APR078  exponent  of  -0.233  for  the 
sole-source  learning  curve. 

-A  percent  loss  on  all  post-competition  production,  recurring 
costs  only,  using  a  learning  curve  exponent  (value  not  given) 
derived  by  IDA79  from  APR073  data.  Presumably  1972  dollars. 

-8  percent  loss  on  all  post-competition  production,  recurring 
costs  only,  using  cumulative  average  price  learning  curve 
exponent  estimated  at  -0.390.  Constant  dollars,  year  not  given. 

-14  percent  loss  on  first  competitive  split-buy  usinq 
learning  curve  exponent  (the  value  is  not  given)  derived  by 
I  DA  7  9  from  APR078  data.  Presumably  1972  dollars. 

Differences  in  savings  estimates  can  result  from  (1)  the  use  of 
different  data,  (2)  differing  statistical  methods,  or  (3)  differing 
definitions  of  what  savings  are.  The  numbers  reflected  in  table  2-1 
were  calculated  by  The  Institute  for  Defense  Analysis  (IDA)  to  show  the 
sensitivity  of  results  to  these  problems.  The  first  two  problems  will 
be  discussed  in  depth  in  the  next  chapter  through  a  review  of  published 
studies.  The  final  one  is  discussed  here. 

The  most  useful  formulation  of  savings  is: 

PVb  -  pvc 

S  -  -  '  '  '  X  IDO!'. 

PV5 
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where 


S  =  percentage  savings 

PV^  =  present  value  of  all  competition  benefits 
PVC  =  present  value  of  all  competition  costs 
PVS  =  present  value  of  post  competition  production  had  the 
system  been  procured  sole-source. 

The  numerator  is  formulated  to  reflect  the  net  impact  to  the  government 
considering  all  cash-flow  impacts  of  competition.  The  denominator 
reflects  what  the  cost  of  the  remaining  program  would  have  been  in  the 
absence  of  competition.  Although  some  studies  have  used  total  program 
cost  as  the  denominator,  precompetition  costs  are  not  relevjnt  for 
decision  making.  They  are  sunk  costs  that  do  not  affect  either  the  ,ole 
source  or  competitive  outcomes. 


Conclusions 

There  exist  a  plethora  of  costs  and  benefits  that  must  be 
considered  in  the  dual -sourcing  decision  itself  and  budgeting  for 
programs  that  are  dual -sourced.  These  include  nonrecurri ng,  recurring, 
and  other  program  specific  impacts  in  the  production  stage  of  the 
program.  Moreover,  competition  might  also  affect  the  cost  of  the 
program's  development  and  deployment  stages.  And  finally,  other 
government  programs  might  also  be  influenced. 

Calculation  of  the  net  impact  of  dual-sourcing  involves  adjustment 
of  historical  data  for  the  impact  of  inflation.  Moreover,  the  casn  flow 
over  time  must  be  discounted  to  reflect  the  time  value  of  money  and  to 
obtain  the  highest  return  on  money  invested  to  develop  the  second 
source.  And  finally,  the  reader  is  cautioned  that  the  savings 

percentage  calculation  varies  in  different  studies  due  to  the 
differences  of  data,  methods,  and  basis  of  savings  calculation. 

Chapter  3  now  turns  to  a  review  of  ex  ante  studies.  Using  the 
criteria  developed  in  Chapter  2,  it  reviews  the  study  data,  methods, 
and  conclusions.  It  also  reviews  published  critiques  of  the  studies. 
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CHAPTER  3 


COMPETITION  STUDIES:  PANACEA  OR  PANDORA'S  BOX? 


Robert  McNamara,  while  secretary  of  defense,  testified  oefore  the 
House  Committee  on  Armed  Services  in  1965  that  when  the  Department  of 
Defense  dual-sourced  weapon  systems  it  saved  an  average  of  25  percent 
And  former  Deputy  Assistant  Secretary  of  Defense  Jacques  S.  Sansler 
testified  before  the  House  Budget  Committee  <i >  recently  as 
8  November  1983  that  "dua 1 -sourci ng  experiments  tried  in  the  past  have 
yielded  an  average  net  program  cost  savings  of  30  percent."^  While 
these  figures  lack  valid  empirical  support,  many  studies  have  been 
published  which  analyze,  ex  post,  the  cost  impact  of  dual  sourcing 
weapons  acquisition  and  posit  a  predictive  model  of  expected  savings  in 
dual-source  decisions.  Other  reports  critique  the  data  and  methods  used 
in  these  studies.  The  balance  of  this  chapter  explores  chronologically 
the  more  substantive  studies  and  the  critiques  by  reviewing  their  major 
data,  methodology,  conclusions,  and  any  model  they  propose  to  predict 
competition  savings  on  future  procurements. ^  It  also  presents  this 
author's  conclusions  regarding  the  predictive  validity  of  these  studies. 
(Usefulness  for  assisting  in  making  future  second-sourcing  decisions.) 


Studies 


E  lectroni  cs  C omnia nd  72 

One  of  the  first  systematic  studies  of  the  effects  of  competition 
was  done  by  the  Army  Electronics  Command  (ECOM)  in  1972. 4  This  study 
compared  the  pre-  and  post-competitive  price  of  13  electronic  items 
having  an  average  price  of  $2,822  and  ranging  between  $290  and  $10,000 
per  item.  The  items  first  entered  production  between  FY  1958  and  FY 
1967  . 


The  methodology  used  was: 


Percent  Savings 


Last  sole-source  price  -  first  competitive  _p_ci_ce 
Last  sole-source  price 


Unit  price  reductions  for  recurring  costs  averaged  53  percent  when 
competition  was  introduced  and  ranged  from  12  to  78  percent.  Savings 
were  attributed  to  all  procurements.  The  study  neither  adjusted  savings 
prices  for  inflation,  nor  considered  nonrecurring  costs.  Nor  did  it 
consider  production  rate  changes  and  learning  from  previous  buys. 
Additionally,  it  did  not  discount  cash  flow. 


The  study  made  multiple  regressions  to  find  a  predictive  model 
using  competitive  lead  time  over  sole-source  lead  time,  competitive 
quantity  over  sole-source  quantity,  and  competitive  delivery  rate 
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(quantity  per  month)  over  sole-source  delivery  rate  as  predictors  of 
price  reductions.  Unable  to  establish  any  relationships,  the 
researchers  concluded: 


To  further  pursue  the  attempts  of  finding  more  significant 
causal  relationships  among  lead  time,  quantity,  and  delivery 
by  regression  techniques  appears  futile.  It  can  almost  be 
concluded  that  the  desired  relationship  is  severely  clouded  by 
other  variables  that  would  be  difficult  to  quantify.  Also, 
the  number  of  .  .  .  items  making  the  transition  from  sole- 
source  to  competition  each  year  is  small,  and  to  accumulate  a 
large  enough  sample  to  provide  sufficient  degrees  of  freedom 
if  the  variable  list  were  expanded  would  span  many 
years. 5 

Notwithstanding  this  caveat,  however,  they  concluded  that  for 
planning  purposes  a  conservative  estimate  of  savings  is  from  25  to  30 
percent . 


Yuspeh  7_3_ 

Larry  Yuspeh  performed  a  1973  study  of  20  items  for  the  Joint 
Economic  Committee's  Subcommittee  on  Priorities  and  Economy  in 
Government. ^  Average  price  of  the  20  items  was  $14,360  in  1970  dollars, 
with  a  price  range  of  $891  to  $92,249  in  inflation-adjusted  dollars. 

The  study  calculated  price  changes  under  competitive  pressures  of 
recurring  production  costs  as  follows: 


Percent  Savings 


Last  sole-source  price  -  first  competitive  pri ce 
Last  sole-source  price 


The  study  found  that  there  was  an  average  of  51  percent  savings  on 
all  programs  analyzed,  with  the  range  of  savings  being  from  16  to  80 
percent.  While  the  researcher  recognized  that  learning  curves  exist,  he 
did  not  factor  a  learning  curve  effect  into  the  savings  estimate.  Data 
are  inflation-adjusted  and  Yuspeh  pointed  out  that  the  learning  curve 
flattens  out  in  competitive  instances.  He  recognized  the  existence  of 
nonrecurring  costs  but  did  not  consider  them  in  offsetting  gross  savings 
in  his  analysis.  He  concluded  that  winner-take-all  awards  resulted  in 
greater  savings  than  did  split  awards. 


While  not  developing  a  predictive  methodology,  he  did  assert  that 
his  report  "establishes  both  the  advantages  and  feasibility  of 
competitive  procurement  of  sophisticated  military  equipment."^ 


IDA  74 


In  1974  the  Institute  for  Defense  Analysis  (IDA)  analyzed 
competitive  procurements  of  19  weapons  systems  with  an  average  price  of 


$6,954  with  prices  ranging  from  $422  to  $87  ,636  in  1970  dolVars.^ 
However,  the  Talos  guidance  and  control  system  cost  $87,636.  Deleting 
this  system  from  the  sample  leaves  the  high  end  of  the  range  at  $6,817. 

Program  savings  were  calculated  using  the  following  methodology: 

Sole-source  price  extrapolated  down  Teaming  curve  - 

Percent  Savings  =  -li^^ompet ^ej)Mce  - - - 

extrapolated  sole-source  price 

The  unweighted  mean  of  recurring  cost  savings  was  36.8  percent,  with  a 
range  from  0.2  percent  cost  increase  to  a  60.8  percent  savings. 

This  study  was  the  first  to  recognize  contractor  learning  by  using 
learning  curve  analysis  in  analyzing  a  repetitive  production  situation 
analysis.  The  study  did  not  consider  nonrecurring  costs  or  production 
rate  changes.  It  did  adjust  data  for  inflation.  IDA  used  multiple 
regression  analysis  to  examine  in  a  predictive  sense  the  effect  on 
post-competition  unit-price  reductions  of  four  variables:  the  exponent 
of  sole-source  learning  curve,  the  ratio  of  competitive  to  sole-source 
quantities,  type  of  competition,  and  number  of  bidders  as  a  measure  of 
the  intensity  of  competition.  Three  statistically  significant 
relationships  were  reported: 

(1)  The  steeper  the  progress-curve  slope  the  less  that  is 
likely  to  be  saved;  (2)  On  the  first  competitive  buy,  maximum 
savings  are  achieved  with  a  buy-out  competition,  minimum 
savings  with  a  50/50  split  (each  competitor  gets  half  of  the 
total);  (3)  Unit-price  reduction  is  ne_£_a_t i vej_y_  correlated 
with  the  ratio  of  the  number  of  units  bought  under  the  first 
competitive  award  to  the  total  number  of  units  produced  under 
all  the  sole-sourced  awards. ^ 

No  statistically  significant  relationship  was  found  for  another 
variable:  the  number  of  bidders.  And  the  study  did  not  develop  a 
regression  equation  for  use  i;i  a  predictive  mode  using  the  statistically 
significant  variables. 


APRQ  78 

The  Army  Procurement  Research  Office  (APR0)  studied  16  Army  and 
Navy  systems  that  had  been  dual-sourced  with  an  average  price  of  $13,138 
in  FY  1972  dollars,  and  the  prices  ranged  from  $589  to  $85,805.  ^ 
Excluding  the  top  three  items,  the  average  price  was  $4,950. 

Savings  were  calculated  as  a  percent  of  total  program  costs  as 
f o 1  lows: 

(Extrapolated  sole-source  learning  curve  price  - 

Percent  Savings  =  -1At-  wmp_eti  U  ve  buy  )  x  compet  i  t  ion  j^^g  tj  ^ 

fotal  Program  Cost 
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Unit  recurring-price  reductions  averaged  10. 8  percent,  ranging  from 
a  net  increase  in  cost  to  the  government  of  13.2  percent  for  the  Mark  46 
Torpedo  to  a  savings  of  51  percent  for  the  Shrike  missile.  The 
methodology  considers  inflation  and  contractor  learning,  and  it  adjusts 
for  contract  changes.  It  does  not  include  production  rate,  nor  does  it 
discount  cash  flows.  The  use  of  total  program  cost  as  a  base  for 
savings  calculations  results  in  lower  savings  calculations  than  other 
studies  because  competition  savings  are  compared  to  total  program  cost 
instead  of  the  proportion  of  the  program  competed  or  the  first 
competitive  buy. 


One  problem  identified  in  the  APRO  78  analysis  involves  using  price 
data  to  develop  learning  curves  when  price  data  can  contain  both 
recurring  and  nonrecurring  costs.  While  the  analysis  uses  other  sources 
of  cost  information  to  filter  out  these  costs,  the  data  are  imprecise. 
For  example,  only  S746.100  is  considered  as  nonrecurring  technical 
assistance  costs  to  the  second  source  from  the  prime  on  the  Shillelagh 
missile.  Others  have  estimated  the  cost  of  technology  transfer  in  the 


millions  of  dollars  on  other  programs 
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The  predictive  methodology  of  the  APRO  78  analysis--cal led  their 
forecasted  savings  methodology  (FSM) .--consists  of  three  parts.  Part 
one  involves  a  competition  screen,  part  two  is  the  forecasted  savings 
estimate,  and  part  three  is  the  competition  index. 


The  competition  screen  consists  of  12  elements  influencing 
competition  against  which  a  potential  dual-source  award  must  be 
evaluated  to  see  if  it  is  worth  pursuing  a  dual-source  strategy.  These 
elements  are  shown  in  table  3-1. 


Table  3-1 

Factors  Influencing  Competition:  The  Competition  Screen 

1.  Prohibitively  high  initial  start-up  costs 

2.  Lack  of  a  definitive  technical  data  package 

3.  Proprietary  data--technology  transfer 

4.  Congressional  i nterests--budget  constraints 

5.  Inadequate  production  quantities 

6 .  Economi c  climate 

7.  Length  of  planned  production  cycle 
3.  Critical  or  scarce  materials 

9.  Nonconformance  to  cost-acco1  nt i ng  standards 
10.  Special  tooling/test  equipment 
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Table  3-l--conti  nued 


11.  Testing  requirements 

12.  Government/i ndustry-wi de  cash  flow  problems 

The  second  step  of  the  FSM  for  those  items  that  pass  the 
competition  screen  is  to  estimate  expected  savings.  Data  relating  to 
savings  from  the  16  systems  in  the  report  were  used  to  obtain  the 
following  predictive  equation:  AUP  =  PUP*9'5  ROQ-*^^ 

Where: 

AUP  =  actual  unit  price  for  all  production  that  occurred  after 
competitive  buy-out 

PUP  =  projected  unit  price  of  sole-source  progress  curve  over  the 
post-competitive  buy-out  production 

ROQ  =  ratio  of  post-competitive  buy-out  production  to  total  program 
production  quantity 

A  competition  index  is  the  third  step  and  involves  an  evaluation 
of  the  qualitative  aspects  that  can  influence  the  amount  of  savings. 
These  aspects  are  listed  in  table  3-2. 


Table  3-2 
Competition  Index 

1.  Perception  of  Competitive  Position 

a.  Production  experience 

b.  Capacity 

c.  Age  of  faci 1 i t i es 

d.  Area  wage  rates 

e.  Union 

2.  Anticipated  Future  Requirements 

a.  United  States 

b.  Foreign  military  sales 

c.  Spinoffs 

d.  Other  components 

3.  F.conomic  Conditions 

a.  Current 

b.  Future 

4.  Company  Goals 

a.  Immediate 

b.  Long-range 

4  b 


Table  3-2--cont i nued 


5.  Risk-Assumption 

a.  Technical  risk 

b.  Quality  of  technical  data  package 

6.  Capital  Investment 

a.  Dollar  value  required 

b.  Use  of  government-furnished  equipment 

c.  Type  of  equipment 

7.  Make-or-Buy  Considerations 

a.  Sole-source  subcontractors 

b.  Government-di rected  subcontractors 

8.  Other 

Types  of  contracts 

Should  cost 

Value  engineering 

Each  factor  is  given  a  weight  from  UO  if  the  factor  has  an 
"extremely  strong  increasing  influence"  on  the  benefits  of  competition 
to  -10  if  the  factor  has  an  "extremely  strong  decreasing  influence,"  or 
each  factor  can  be  assigned  any  weight  between  the  extremes.  A  weighted 
average  of  the  individual  factor  scores  based  on  their  importance  to  the 
specific  system  is  calculated  and  a  subjective  judgement  made. 


IDA  79 


The  Institute  for  Defense  Analysis’  second  study  was  published  in 
1979  and  summarized  31  cases  from  the  three  studies  just  discussed.^ 
Cumulative  average  unit  price  was  $17,675,  with  prices  ranging  from  $138 
to  $177,622. 

Savings  were  calculated  as  follows: 

Sole-source  price  extrapolated  down  learning 
curve  -  contract  price 

Percent  Savings  = - - - — - 

Projected  Sole-Source  Price 

This  study  estimated  savings  averaged  35.1  percent  on  all  post 
sole-source  buys  and  ranged  from  a  23  percent  cost  increase  to  a  savings 
of  64  percent.  It  did  not  consider  discounting,  did  not  make  rate 
adjustments,  and  considered  only  some  of  the  nonrecurring  costs.  While 
data  were  inflation-adjusted,  the  methodology  was  not  included. 
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The  study  developed  a  forecasting  model  as  follows: 

-  ( 1  *-FQ/SSQ)sss  -  .414 

FSFB  =  - 

l-( 1+FQ-SSS-l  )-SsS-l 

Where: 

FSFB  =  fractional  savings  on  future  buy 

FQ  =  quantity  procured  competitively 

SSQ  =  sole-source  quantity 

SSS  =  sole-source  learning  curve  slope 

The  equation  is  the  cumulative  average  price  variant  of  the  standard 
learning  curve  equation.  Gross  savings  on  competitive  quantities  are 
predicted  as  a  function  of  the  ratio  of  total  quantity  to  sole-source 
quantity,  the  known  sole-source  slope  derived  from  cumulative  average 
prices,  and  the  mean  (-.414)  of  competitive  slopes  from  its  sample  of  31 
items. 

The  study  concluded  that  the  savings  forecasting  model  is  only  a 
moderately  successful  predictor  of  actual  savings.  It  stated,  however, 
that  for  planning  purposes,  projections  of  post  competition  savings  are 
10  percent  for  split  award  buys  and  20  percent  for  buy-outs.  And  they 
also  add  this  caveat  to  their  findings: 

It  is  in-fact  likely  that  no  precise  and  stable  predictive 
relationships  exist;  there  are  so  many  dimensions  of  variation 
surrounding  each  procurement  ( e . g . ,  technology,  market 
conditions)  that  each  system  is  to  a  considerable  extent 
unique. 


TASC_7_9 

In  1979  The  Analytic  Sciences  Corporation  (TASC)  was  selected  by 
the  Joint  Cruise  Missile  Project  Office  to  develop  a  method  to  estimate 
comparative  production  costs  for  sole-source  and  competitive  dual-source 
production  strategies. TASC  reanalyzed  45  procurements  that  had 
previously  been  reported.  In  contrast  to  the  previous  studies,  which 
characterized  the  reduced  cost  as  a  shift  down  in  the  learning 

curve  under  competitive  circumstances,  TASC  characterized  the  savings  as 
an  immediate  downward  shift  in  the  learning  curve  at  the  point  of 

competition,  due  to  cost-and-prof i t  reduction,  and  a  rotation  or 
steepening  of  the  curve  as  the  previously  sole-source  producer 
introduces  efficiency  into  the  production  process  as  shown  on 
figure  3-1.  The  standard  power  function  was  used: 
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LOG  OF  CUMULATIVE  UNITS  PRODUCED 


Z  =  AX8 


Where: 

Z  =  unit  cost  of  the  Xth  item  produced 
A  =  first  unit  cost 
X  =  cumulative  quantity  produced 
B  =  parameter  for  slope  of  the  quanti ty/cost  curve 


Savings  were  calculated  as: 


Percent  Savings 


Sole-source  price  extrapolated  down  learning 
cur ve_-_  f  i  rst  c omp e t  i  t_i_v e  p rice 

Sole-Source  Price  Extrapolated 


The  study  estimated  the  average  gross  savings  for  recurring  costs  from 
both  the  shift  and  rotation  effects  at  33  +  6  percent.  TASC  also  found 
that  a  combined  prof i t-and-cost  reduction  [shift)  of  12  +_  2  percent  was 
probably  too  low,  and  estimated  an  improvement  (rotation)  of  S  ^  2 
percent.  The  analysis  was  i nf  1  at i on-adjusted ,  using  an  unspecified 
index,  but  did  not  consider  production  rate.  The  reported  recurring 
cost  savings  ranged  from  an  increase  in  cost  to  the  government  of  lb. 2 
percent  to  a  savings  of  67.7  percent. 


TASC  81 

In  1981  The  Analytic  Sciences  Corporation  published  a  further  study 
that  projected  comparative  sole-source  and  competitive  dual -source 
production  cost  estimates  for  sea-launched  and  ground-launched  cruise 
missile  airframes.^  TASC  modified  its  1979  analysis  and  included 
projected  impacts  of  both  production  rate  and  total  production  quantity 
(learning  curve)  variation. 

The  model's  basic  formulation  is:  Z  =  AX“  Y  — 

Where: 

Z  =  unit  cost  of  the  Xth  item  produced 
A  =  1st  unit  cost 
X  =  cumulative  quantity  produced 

B  -  parameter  for  slope  of  the  quant i ty/cost  curve 
Y  -  some  measure  of  production  rate 
C  =  parameter  of  slope  of  the  rate/cost  curve 
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The  study  developed  estimates  of  the  parameters  for  five  tactical 
missile  systems  using  a  least-squares  fit  of  the  exponential  formulation 
to  observed  data.  The  methodology  involved  finding  the  solution  to  a 
set  of  nonlinear  equations  in  five  unknowns.  It  assumed  successive 
iterations  from  an  initial  starting  point  to  minimize  squared  error 
using  Newton's  method  for  finding  the  roots  of  a  nonlinear  function.^ 
It  calculated  learning  curves  using  cumulative  units,  and  it  used  lot 
size  as  a  proxy  production  rate.  Using  this  procedure,  which  was 
inexactly  specified  in  its  report,  the  study  made  parameter  estimates 
as  shown  in  table  3-3.  In  making  these  calculations,  the  study  assumed 
the  shift  and  rotation  of  the  curve  will  not  occur  until  the  point  of 
competition. 

Those  making  the  study  contend  their  second  model,  which  includes 
the  effects  of  product i on-rate  variations,  is  substantially  different 
f^om  their  first  model,  which  does  not  allow  for  production-rate 
effects.  Therefore,  they  stated  that  comparing  parameters  between 

models  is  essentially  meaningless.^ 

They  developed  a  predictive  theoretical  framework  which  they 
described  as  "both  theoreti ca 1 ly  and  intuitively  pleasing."  It  is 

represented  in  figure  3-2.^  Curve  S}  depicts  the  cost  improvement 
curve  of  the  sole-source  producer  (with  variations  due  to  production 

rate  removed)  in  a  noncompetitive  environment.  Curve  shows  the  shift 
and  rotation  of  the  cost-improvement  curve  after  competition  is 

introduced.  Curve  S3  is  a  theoretical  construct  of  the  optimal  or 

best-cost  improvement  curve  one  might  observe  if  the  manufacturer  were 
under  continuous  competitive  pressure  from  the  outset.  It  is  obtained 
by  beginning  with  the  historically  derived,  noncompetitive  first-unit 
cost  and  ending  when  che  curve  achieves  parity  with  the  competitive 

last-unit  cost.  It  represents  what  TASC  states  might  have  happened  had 
tne  original  producer  been  under  continuous  competitive  pressure  from 
the  outset.  TASC  has  developed  separate  curves  for  both  missile  and 
electronic  systems. 

TASC  researchers  use  their  model  in  a  predictive  mode  by  using  a 

proxy  production  rate  and  program-speci f i c,  sole-source  learning  curve 

data.  While  they  did  not  specify  their  formulation,  it  appears  to  be  as 
f  0 1 1  ow  s : 

7  =  A  [exp  (d-OC)l  X  fb  x  f-DC)fc. 

Where : 

2 ,  A ,  X ,  Y  are  above 

DC  equals  binary  with  a  value  of  one  when  lot  is  procured  under 

competition  and  zero  otherwise. 

Using  this  formulation,  they  assume  that  the  second  source  begins 
producing  at  the  same  first-unit  cost  as  does  the  first,  producer,  but 


UNIT 

COST 


Q,  Q2 

QUANTITY  PRODUCED 

Figure  3-2.  Cost  Improvement  Curves  with  Variations 
Due  to  Production  Rate  Removed 


that  his  cost-improvement  rate  is  four  percent  steeper  than  the  first 
source.  They  then  assume  that  at  the  point  of  competition  a  shift  and 
rotation  would  be  observed  on  the  cost-improvement  curve  of  the  first 
source.  The  magnitude  of  the  shift  and  rotation  is  assumed  to  be 
sufficient  for  the  company  to  eventually  reach  cost  parity  with  the 
optimum  cost-improvement  curve.  (Point  Q2  in  figure  3-2  is  derived  from 
historical  data  using  the  TASC  formulation.) 

This  TASC  study  does  not  consider  nonrecurring  costs,  nor  does  it 
discount  savings  over  time.  Additionally,  it  does  not  consider 
potential  impacts  on  other  programs. 


SAI  82 


In  1982,  Michael  Beltramo  and  David  Jordan  of  Science  Applications, 
Incorporated  (SAI)  conducted  a  three-part  study  for  the  Navy  on  the 
impact  of  dual  sourcing.^  They  reanalyzed  seven  missile  and  bomb 
systems  that  had  previously  been  analyzed.  Sole-source  costs  were 
estimated  by  projecting  down  the  sole-source  learning  curve.  To  show 
the  sensitivity  to  percent  savings  calculation  methods,  they  calculated 
savings  using  three  cases: 

S2 


Case  one--The  first  competitive  lot  price  expressed  as  a  percentage 
of  estimated  sole-source  lot  price. 

Case  two--The  total  competitive  recurring  production  cost  with 
estimate  sole-source  cost. 

Case  three--The  estimated  savings  for  total  program  by  toe 
percentage  of  the  total  quantity  completed. 

Using  this  methodology,  the  study  considers'!  rne  nonrecurring 
costs  with  the  results  shown  in  table  3-4.  Differences  in  estimated 
costs  or  savings  of  the  three  cases  range  from  20.5  to  27.5  Dercent 

additional  costs  to  the  government  over  the  sole-source  estimated 

amounts,  depending  on  the  calculation  methodology. 

The  study  did  not  consider  production  rate  nor  did  it  discount  cash 
flows  over  time.  Some  nonrecurring  costs  were  considered,  and  some 

government-related  costs,  while  acknowledged,  were  not  quantified. 

The  authors  did  not  outline  a  predictive  model,  but  they  did 
qualitatively  explore  the  following  six  criteria  as  questions  to  ask 
when  considering  a  program  for  dual-sourcing: 

1.  Does  the  government  own  the  data  rights? 

2.  Is  the  item  to  be  produced  technically  complex  with  respect  to 
design  and  manufacturing  processes? 

3.  If  the  item  is  complex,  would  technical  assistance  be  required 
from  the  initial  source  to  establish  an  effective  second  source? 

4.  What  is  the  total  required  quantity?  How  many  units  would  be 
produced  by  the  initial  source  prior  to  technology  transfer?  Prior  to 
dual-source  competition?  How  many  units  would  the  second  source  produce 
during  educational  buys? 

5.  What  is  the  maximum  quantity  to  be  produced  in  a  year?  Can 
production  efficiency  be  optimized  at  half  of  that  quantity? 

6.  Does  the  initial  source  have  traditional  rivals  for  the  iten  to 
be  produced?  Who  are  they? 


Greer  und__L_i_ao  33 

Previous  studies  quantified  savings  and  attempted  to  correlate 
savings  with  a  number  of  parameters:  sole-source  learning  curve  slope, 
percentage  of  total  program  completed,  pro  Let  complexity,  and  other 
predictive  variables.  In  addition,  Yuspeh  73  and  IDA  79  addressed  the 
differences  in  savings  between  split-buy  and  buy-out  strategies,  hone, 
however,  identified  specific  causative  variables;  that  is  those  that 


Table  3-4 

Sparrow  AIM-7F  Actual  vs  Estimated  Life  Cycle  Costs 
Helui.ud  to  Dual-Source  Competition  (millions) 


S4 


Does  not  include  government  costs  which  are  probably  higher  for  dual -source  competition. 


caused  the  difference  of  magnitude  of  savings  or  loss  on  a  program. 
Willis  Greer  and  Shu  Liao  attempted  to  measure  the  correlation  of  the 
magnitude  of  savings  with  contractor  "hungri ness . As  a  measure  of 
hungriness,  they  used  capacity  utilization  for  the  aerospace  industry 
and  correlated  this  measure  with  savings  or  loss  on  missile  programs. 
Their  models  are  formulated: 

Production  Rate  Model:  P  =  KQaRb 

Capacity  Utilization  Model:  P  =  KQa  Uc  exp(dm)  exp(fn) 

Where 

P  =  average  price 
Q  =  midpoint  quantity  of  buy 

U  =  smoothed  utilization  percentage  for  industry 
M  =  1  if  the  buy  is  under  dual-sourcing,  0  otherwise 
N  =  1  if  the  competition  is  wi nner-takes-a 1 1 ,  0  otherwise 
a,  b,  c,  d,  and  f,  are  parameters 

Note  that  the  second  model  substitutes  capacity-utilization  for  the 
production  rate.  Greer  and  Liao  concluded  that  the  capacity-utilization 
model  is  a  better  predictor  of  savings  with  the  following  parameter 
values  and  nonsignificant  variables  deleted: 

For  the  original  source: 

With  median  values: 

p  =  kq-0.278u  +  1.250e  -0.101Me  -0.854N 
With  mean  values: 

P  =  kq-0.260u  +  1.765e  -0.201Me  -0.8S4N 
The  second  source  models  are: 

With  median  values: 

P  =  KQ-0.174e  --520N 

The  study  concluded  that  it  is  best  to  compete  when  capacity 
utilization  is  below  80  percent,  as  shown  in  their  data  arrayed  in  table 
3-5: 


Table  3-5 

Relation  of  Savings  and  Capacity  Utilization 


Annual 


P  rocu  rement 
_ Program 

Percent  Savings 
or  (Loss)  Due 
to  Competition 

Average  Capacity 

Ut i 1 ization  During 
Dual-Source  Phase 

TOW 

26.0 

63.5 

Rockeye  Bomb 

25.5 

70.9 

Bui  lpup  AGM-12B 

18.7 

76.2 

Shillelagh  Mi ss i 

le 

(4.7) 

87.0 

Sparrow  AIM-7F 

(25.0) 

81 .6 

MK-45  Torpedo 

(30.9) 

91 .6 

Sidewinder  AIM-90 /G 

(71.3) 

82.3 

The  major  problem  with  the  study  by  Greer  and  Liao  is  the 
forecasting  of  capacity  utilization.  For  example,  they  forecasted 

capacity-utilization  data  for  the  seven  systems  they  examined  and,  using 
their  predictive  model  decision  rules,  made  the  wrong  decision  in  three 
of  seven  cases.  In  other  words,  they  applied  their  model  to  the  seven 

systems  outlined  in  table  3-5  and  predicted  an  outcome  different  from 
what  actually  happened  in  three  of  seven  cases. 

They  also  found  that  returns  earned  by  contractors  on  DOD  business 
are  measurably  lower  than  the  returns  on  commercial  business  during 
periods  of  low-capacity  utilization.  Moreover,  they  concluded  that 
competition  produces  greater  savings  when  firms  are  "hungry."  They 
concluded  that  dual -sourci ng  is  of  little  benefit  as  a  cost  reducer  when 
industry  is  very  active. 


BeJ  tramo  84 

Beltramo  prepared  a  follow-on  paper  from  his  1982  effort  (SAI 
82).  In  it,  he  drew  a  distinction  between  procurements  with 

winner-take-all  awards  and  competitive  split-buy  awards.  He  presented 
the  data  in  table  3-6  for  winner-take-all  awards,  calculating  savings 
using  his  three  cases  discussed  earlier.  Table  3-7  presents  data 
regarding  competitive  split-buy  awards. 

fable  3-6  shows  that  when  wi nner-take-a 1 1  competitions  are  used, 
the  majority  of  the  samples  show  savings  regardless  of  which  case  method 
is  used  for  calculation.  In  contrast,  as  shown  in  tabl°  3-7,  four  of 
the  seven  cases  show  a  loss. 
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Table  3-6 

elected  Cost/Quanti  ty  Data  f  or  W  i  nner-Take-A  11  Competitions 
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*Actual  competitive  split-buy  costs  divided  by  extrapolation  of  initial  sole-source  learning 
curve  for  competitive  split-buy  quantity. 


Using  these  previously  calculated  data,  Beltramo  concluded  t-at 
competitive  split  buys  have  resulted  in  both  higher  and  lower  recurring 
production  costs  than  would  be  incurred  by  sole-source  producers.  he 
also  stated  that  winner-take-all  competitions  should  be  held  wnennv'r 
possible. 


Oi  s_cu  s_s_icm 

The  first  part  of  the  chapter  discussed  the  more  prominent  studies 
completed  on  the  impacts  of  dual-sourcing  weapons  systems  in  production. 
Very  little  new  data  have  been  developed  by  these  studies  since  the  45 
systems  analyzed  in  previous  studies  were  reanalyzed  in  the  TASC  7 9 
report.  The  balance  of  this  chapter  compares  the  conclusions  reached  in 
each  of  the  studies.  It  also  critiques  their  predictive  models  for  use 
by  DOD  in  dual -sourci ng  decisions  by  examining  the  elements  of  cost 
impact  that  each  study  considered. 


EJejrients  Con  side  red 

Table  3-8  arrays  the  elements  of  cost  and  benefits  identified  in 
chapter  2  as  potentially  impacting  on  the  net  cost  (added  costs  minus 
benefits)  accruing  to  the  government  by  dual -sourci ng  weapons  in 
production.  It  also  identifies  the  studies  discussed  above  and  the 
elements  that  each  study  considered  in  its  analysis. 

The  studies  have  become  more  comprehensive  in  the  elements  they 
consider  and  have  contributed  to  our  knowledge  about  dual -sourcing. 
However,  none  of  the  studies  has  considered  all  of  the  potential 
dual-sourcing  costs  and  benefits.  Most  studies  have  explicitly 
considered  recurring  production  costs  and,  to  a  limited  extent, 
have  considered  nonrecurring  production  costs  where  data  were  readily 
avai lable. 

However,  thev  have  seldom  identified  impacts  on  the  other  weapon 
system  program  stages.  Any  benefits  accruing  to  the  government  during 
spare  parts  acquisition  due  to  nonavailability  of  technical  data  were 
not  addressed.  Additionally,  no  study  identified  the  impact  on  other 
programs  that  a  change  in  overhead  rates  would  have  because  of  the 
altered  allocation  bases  in  the  prime  and  second  sources. 

The  other  major  area  where  costs  were  omitted  were  those  incurred 
directly  by  the  government.  These  include  source  selection,  second- 
source  development,  program  management  costs,  extra  contract 
administration  costs,  and  the  extra  costs  associated  with  configuration 
management . 

Perhaps  the  biggest  omission  from  all  studies  that  claimed  savin; s 
was  that  the  time  value  of  money  was  not  cons  idered--the  out-vem 
savings  were  not  discounted  to  a  common  base-year.  This  om’ss’an 
precludes  any  accurate  assessment  of  what  the  "true"  cost  or  savings  is 
to  the  government  when  the  decision  to  dual -source  is  made  f  cost- 
reduction  purposes. 
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Table  3-8 


Clients  Consi  derod  by  Study 


ECOM  72 

YUSPEH  73 

IDA  74 

APRO  78 

IDA  79 

TASC  79 

TASC  81 

C\J 

co 

<* 

on 

Quantifiable 

Nonrecurring 

Second -Source  Selection 

N 

N 

N 

N 

N 

N 

N 

N 

Second -Source  Development 
Technical  Data  Pkg 

Purchase 

N 

N 

N 

N 

N 

N 

N 

Y 

Validation 

N 

N 

N 

N 

N 

N 

N 

Y 

Special  Tooling 

N 

N 

N 

N 

N 

N 

N 

Y 

Special  Test  Equip 

N 

N 

N 

N 

N 

N 

N 

Y 

Tech  Transfer 

N 

N 

Leader-Fol lower 
Education  Buys 

N 

N 

N 

N 

N 

N 

Some 

N 

N 

N 

N 

N 

Y 

Govt  Monitoring 

N 

N 

N 

N 

N 

N 

N 

N 

First  Article 

N 

N 

N 

N 

N 

N 

N 

Y 

Contingent  Liabilities 

N 

N 

N 

N 

N 

N 

N 

N 

Recurring 

Cost/Quantity  Relationship 

N 

Production  Rate 

N 

N 

N 

N 

N 

N 

Y 

Learning 

N 

N 

Y 

Y 

Y 

Y 

Y 

Y 

Contract  Administration 

N 

N 

N 

N 

N 

N 

N 

N 

Configuration  Mgt 

N 

N 

N 

N 

N 

N 

N 

N 

Other  Costs 

Company- Funded  R&D 

N 

N 

N 

N 

N 

N 

N 

N 

Spare  Parts 

N 

N 

N 

N 

N 

N 

N 

N 

Overhead  Impact 

N 

N 

N 

N 

N 

N 

N 

N 

Logistics 

N 

N 

N 

N 

N 

N 

N 

N 

Prof i t 

N 

N 

N 

N 

N 

N 

N 

N 

Nonquanti f iabl  e 

Productivity 

N 

N 

N 

N 

N 

N 

N 

N 

N 

Product  Qua! ity 

N 

N 

N 

N 

N 

N 

N 

Cooperation 

N 

Technical  Transfer 

N 

N 

N 

N 

N 

N 

N 

Ongoi ng 

N 

N 

N 

N 

N 

N 

N 

N 

Time  Delays 

N 

N 

N 

N 

N 

N 

N 

N 

Cl  a ims 

N 

N 

N 

N 

N 

N 

N 

N 

Statistical 

Inflation  Adjust 

N 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Discounting 

N 

N 

N 

N 

N 

N 

N 

N 

fau 


N  =  N0  Y  =YES 


Savings  Percentages 


Table  3-9  arrays  by  category  of  system  and  program  the  results  of 
data  analysis  and  reanalysis  by  each  study  relating  to  the  cost  and 
benefit  factors  that  were  considered. ^  Two  conclusions  can  be  drawn 
from  a  perusal  of  the  data.  First,  within  a  specific  study  there  is 
a  range  of  different  savings  or  losses  across  programs.  Second,  the 
studies  reach  different  conclusions  on  the  impact  of  dua 1 -sourci ng,  both 
in  the  analysis  of  the  same  weapon  system  and  the  average  oroo 
savings  calculated  by  each  study. 


Results  Within  Studie s 

The  pattern  of  savings  estimates  within  any  of  the  studies  shows  a 
wide  range  of  savings.  This  pattern  persists  when  all  systems  estimates 
contained  in  the  study  are  arrayed.  More  revealing,  however,  is  tnat 
when  similiar  systems  such  as  missiles  or  electronics  are  grouped,  this 
same  range  of  estimates  persists. 

Another  problem  is  that  means  are  unweighted  means  that  can  be 
misleading  when  comparing  systems  of  widely  differing  costs.  As  an 
example,  with  unweighted  averages,  a  5  percent  savings  on  a  SI  item  is 
weighted  the  same  as  a  5  percent  loss  on  a  $1,000  item.  On  the  average, 
these  two  impacts  result  in  no  average  change,  even  though  the  monetary 
impact  is  obvious.  Also,  an  examination  of  the  mean  in  relation  to  the 
median  shows  that  the  underlying  distribution  of  values  in  each  group 
can  differ.  With  the  median  to  the  left  of  the  mean,  the  distribution 
is  skewed  right.  If  the  median  rests  to  the  right  of  the  mean,  the 
distribution  is  skewed  left. 

Finally,  as  discussed  in  the  preceding  section,  these  figures  do 
not  represent  the  net  impact  to  the  government  of  dual -sourci ng.  Taken 
together,  these  concerns  cast  a  shadow  of  caution  on  the  use  of  any 
rule-of-thumb  estimate  of,  for  example,  a  10-percent,  25-percent  or 
other  fixed-percentage  savings  available  when  systems  are  competed. 


Results  Between  Studies. 

Analysis  of  the  same  procurement  by  different  analysts  has  resulted 
in  different  estimates  of  impacts.  In  some  instances,  as  in  the  case  of 
the  Shillelagh,  some  analysts  attribute  a  savings  while  others  attribute 
a  loss.  These  differences  occur  because  different  methodologies  are 
employed  and  different  elements  of  cost  and  benefits  are  evaluated.  And 
they  show  that  there  is  room  for  argument  among  competent  analysts  over 
the  impact  of  dual-sourcing  on  competed  programs. 

The  differences  can  occur  because  different  studies  use  different 
indices  or  base-years  to  adjust  data  for  inflation.  For  example, 
adjusting  contract  data  to  constant-year  dollars  using  the  consj-np’- 
price  index  instead  of  the  wholesale  price  index  could  lead  to  different 
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Table  3-9 

Estimated  Percentage  Savings  or  (Loss) 
Due  to  Competition 


FCOM 

72 

YUSPKH 

73 

IDA 

74 

APRO 

78 

IDA 

79 

SAT 

84 

Missiles  &  Missile  Components 

TOW 

48.1 

8.5 

8.9 

12.3 

22.6 

DRAGON  Round 

2.7 

2.8 

SHILLELAGH 

68 

(0.2) 

5.9 

(8.0) 

9.4 

(6.3) 

TALOS  (CSC  unit) 

42 

42.  3 

40.8 

39.8 

BULLPUP  12  (Martin) 

25 

1  3.9 

31.7 

26.5 

25.8 

SIDEWINDER  AIM-9D/C 

(4.6) 

0.  7 

(22.0) 

SIDEWINDER  AIM-9B 

1.6 

(5.6) 

Standard  Missile  MR 

RIM  66A 

60 

(4.2) 

59.2 

(2.9) 

Standard  Missile  ER 

RIM  67A 

59 

34.0 

HAWK  Motor  Parts 

50 

6 . 4 

45.7 

49.9 

4b.  7 

TOW  Launcher 

30.2 

44.2 

30.2 

DRAGON  Tracker 

12.0 

12.3 

SPARROW  AIM-7F 

(G&C  Unit) 

16.0 

(20.5) 

MEAN 

50.7 

26.0 

11.9 

17.3 

22.6 

6.2 

MEDIAN 

54.5 

J  3.9 

8.  5 

8.9 

19.4 

2.9 

Torpedos  &  Bombs 

MK-48  Warhead 

54 

53.2 

48.6 

50.9 

MK-48  Electric  Assembly 

55 

37.5 

47.0 

24.9 

MK-48  Exploder 

80 

61.2 

MK-48  Test  Set 

79 

61  .8 

Rockeye  Bomb 

19 

(23.0) 

(4.5) 

1.7 

MK-46  Airframe  &  G6C 

(  16.4) 

MEAN 

57.4 

(23.0) 

42.8 

10.8 

MEDIAN 

55 

(23.0) 

48.6 

14.3 

Electronic  Components 

FAAR  Radar 

16.6 

16.6 

FAAR  TADDS 

18.2 

18.2 

AN/ARC-131 

(2.1) 

(16.2) 

UPM-98  Test  Set 

44 

J.O 

11.5 

PP-4  763/GRC  Power  Sup 

58 

.  J 

.  5 

TD-204  Cable  Combiner 

50 

50.2 

62.1 

42.9 

5!  .2 

TD-202  Radio  Combiner 

31 

52.5 

4b.  8 

40.2 

51.1 

TD-352  Multiplexer 

58 

57.8 

58. 0 

55.6 

58 

TD-660  Multiplexer 

41 

30.2 

38.3 

28.4 

35.9 

60-6402  Elec  Control 

56 

57.0 

49.4 

52.7 

SPA-66  Radar  Indicator 

16 

(  1.4) 

APX72  Airborne  Transponder 

40 

32.  b 

2  7.1 

23.  1 

*28.4  >r 

AN/ ARC- 54 

75 

55.0 

b  j.  1 

AN/PRC-77 

48 

*4.8 

20.3 

4  1  .  9 

24 . 2 

AN/GRC-lOb 

34 

4  \ .  3 

41.8 

AN/GRC-10  i 

59 

>8.  7 

bO.  1 

3  1.8 

AN/APM-1 2  3 

67 

hi  .2 

67.7 

SPA- 2 5  Radar  Indicator 

21.1 

48.8 

10.7 

7  3.1 

l’SM-181  Test  Set 

3b.  n 

Sh.O 

36.3 

FGC-20  Teletype 

12.0 

2  1.7 

39.9 

28.8 

MD-522  Mod/Demod 

37 

HO.  3 

>8.  b 

>1.0 

3  3.0 

CV-1348  Signal  Conv 

59 

*>1.7 

1 5  4.0 

43.4 

b  i  .  0 

AN/ARA-63  Radio 

b  \ 

37.9 

AN/SQS-23  208A  T  nsd 

4  7 

*2.  3 

AN/PKL-25 

3  3.0 

AN/ AS N- 4 3 

12 

10.7 

AN/FYC-8X 

4  3.2 

MK-980/PPS-5 

<3  5 

3b  .  0 

b  b  .  3 

PRT-4 

41 

-«2 .  3 

Aern>'  42-0750 

34.8 

38.  1 

At-rne  42-2028 

14.4 

14.1) 

MEAN 

HfSg 

4  3.9 

11.8 

4  7.2 

3  3.3 

4  4  .  1 

MEDIAN 

B A 

■  m 

30.2 

4  .  9 

32.  1 

4  1 

M.2 

TOTAL  MEAN 

5  1.  1 

49.  7" 

m.o 

n  .8 

3  l.S 

3  2.9 

2  7 .  3 

TOTAL  MEDIAN 

3  7  .  3 

32 

54.9 

8. 

44  1  .  H 

14.9 

29 . 0 

*  Commonality  between  7859  and  7859A  is  at  issue:  if  common,  total  savings;  if  not, 
total  loss. 


b(\ 


results  to  the  extent  the  indexes  are  different.  Otner  difference-,  can 
originate  through  use  of  different  estimatin')  techniques  or  dat  i  k  i',f,s. 
Finally  the  differing  savings  calculations  as  discussed  in  chapter  V  -a' 
lead  to  diffen>nt  results,  ceteris  paribus. 


Critiques 

Several  published  studies  critique  these  studies  in  detail. 
have  been  done  by  such  consulting  organizations  as  the  Institute  *;r 
Defense  Analysis,  Science  Applications  Incorporated,  RAND,  and  toe 
Logistics  Management  Institute. 

IDA__7_9_.  The  IDA  79  report  discussed  earlier  reviewed  the  ECOM  72, 
IDA  7T,  and  APRO  73  studies. ^  Its  major  concern  with  ECOM  72  was  the 
failure  to  fit  progress  curves  to  the  sole-source  production  lots  prior 
to  computing  savings.  IDA  79  fit  progress  curves  to  the  ECOM  data  and 
identified  savings  of  53  percent  vice  the  56  percent  identified  by  ECOM. 
It  concluded  that  the  systems  selected  by  ECOM  were  exceptionally  good 
candidates  for  the  introduction  of  competition. 

Concerns  with  the  IDA  74  study  related  to  its  failure  to  separate 
nonrecurring  from  recurring  costs  and  to  the  fact  that  the  sample  could 
not  be  considered  representat i ve  because  almost  all  systems  were 
formally  advertised  with  more  than  two  bidders.  It  stated  that  the  -VRi) 
73  study  was  the  most  sophisticated  and  well  developed  methodology  for 
estimating  savings.  However,  it  felt  that  extreme  data  manipulation  and 
extrapolation  were  required  because  APRO,  due  to  a  lack  of  sufficiently 
reliable  data,  extrapolated  progress  curves  to  ten  or  more  times  the 
sole-source  quantities  to  which  they  were  fitted.  IDA  79  was  a’so 
concerned  about  the  impact  that  actual  or  imminent  price  competition  had 
on  sole-source  price.  And  it  expressed  concern  about  calculating 
savings  as  a  percentage  of  the  total  program  rather  than  post- 
competition  production.  It  believed  that  this  method  understates  toe 
savings  to  be  expected  on  future  competitive  unit  production  costs. 

IDA  pointed  out  the  sensitivity  of  savings  estimates  to  different 
methods  of  estimating  progress  curves  and  the  inclusion  of  different 
cost  impacts.  Table  3-10  reflects  their-  analysis  of  APRO  data  and  how 
including  different  elements  and  using  different  methodologies  for 
calculating  savings  can  change  savings  estimates. ^ 

Percentage  savings  for  each  system  as  estimated  by  APR..’)  are 
presented  in  column  one  of  the  table.  The  estimates  in  column  two  and 
column  three  follow  the  APRO  methodology  insofar  as  available  fixed 
costs  were  used  to  determine  savings  and  savings  were  expressed  as  a 
percentage  of  total  program  costs.  However,  the  learning  curve  used  t i 
make  the  projections  of  the  sole-source  production  costs  for  toe 
estimates  in  column  two  w°re  fitted  witn  a  weighted  regression  net  no  i. 
Instead  of  treating  each  oroduction  lot  equally,  as  was  don-^  for  the 
regression  curve  used  in  column  on11,  the  production  lots  were  w.> :  j-*  n 
in  proportion  to  the  sixe  of  the  lots.  Col  inn  three  learning  curves 


were  estimated  by  usinq  cumulative  average  price  rather  tnan  ave-ag-1 
price  of  each  production  lot  as  the  dependent  variable.  The  progress 
curves  for  columns  two  and  three  were  estimated  using  only  available 
data  on  sole-source  production  lots.  However,  APRO  used  something 
than  sole-source  production  lots  for  10  of  the  13  systems  analyzed. 

For  columns  four  through  six,  the  progress  curve  as  provided  by 
APRO  was  used  to  derive  the  estimates  except  as  specified  otherw1 se  in 
footnotes  for  the  TOW  missile.  The  actual  fixed-costs  incurred  in 
introducing  competition  were  neglected  in  calculating  the  estimates  in 
these  three  columns;  savings  were  calculated  for  recurring  production 
costs  only.  Savings  for  each  system  were  estimated  for  a1’  past- 
competition  production  in  column  four;  the  first  split-buy  competition 
awards  in  column  five;  and  the  first  competitive  buy-out  award  in 
column  six.  Finally,  the  estimates  of  savings  on  the  first  competitive 
buy  as  reported  in  IDA  74  are  given  in  column  seven  for  systems  included 
in  both  sides. 

In  referring  to  the  APRO  predictive  model,  IDA  concluded  that  the 
two  explanatory  variables,  projected  price  and  ratio  of  post-competition 
quantity  to  total  program  quantity,  are  insufficient  to  predict 
accurately  the  actual  savings  resulting  from  introducing  competition. 
It  concludes  with  the  general  assessment  that  these  models 

are  insufficient  for  determining  government  policy.  .  .  .  We 
are  forced  to  make  inferences  about  the  potential  impact  of 
introducing  competition  ...  on  the  basis  of  a  sample 
which  cannot  be  regarded  as  random  and  representative  .... 

The  estimated  savings  for  any  particular  system  is  therefore 
subject  to  considerable  error. ^ 

RAND 81.  RAND  conducted  a  1981  study  for  the  Office  of  the  inder 

Secretary  of  Defense  for  Research  and  Engineering.^  It  reviewed  four 
studies:  ECOM  72,  IDA  74,  APRO  78,  and  IDA  79.  It  stated  that 

these  studies  contribute  to  our  understanding  of  the 
competitive  reprocurement.  process,  but  they  do  not  (with  toe 
possible  exception  of  electronic  items)  provide  convincing 
evidence  of  savings  due  to  competitive  reprocurement,  nor  do 
they  provide  reliable  quantitative  tools  for  decision-making.-7 

The  study  highlighted  the  fact  that  savings  estimates  made  by 
different  analysts  on  a  single  system  often  vary  considerably  as  in  the 
case  of  Shillelagh  which  could  range  from  79  percent  to  -14  percent..^'’ 
It  also  pointed  out  that  there  is  evpn  disagreement  in  the 
rank -orderi ngs  of  savings  by  program  as  shown  in  table  3-11.  RAN.)  81 
stated  that  the  major  difference  between  APRO  and  IDA  79  was  that  the 
former  used  unit  production  costs  and  th<j  latter  cumilative  average 
prices,  and  it  expressed  concern  that  neither  used  discounting  in 
their  calculations.  It  concluded  that,  the  existing  body  of  analysis  nas 
not  provided  an  adequate  set  of  management  tools  for  estimating 
either  the  benefits  or  the  costs  of  competitive  reorocurement.  It  also 


concluded  that  it  would  "be  an  understatement  to  say  that  the 
determinants  of  post-competitive  price  differences  have  not  yet  been 
identified;  we  were  unable  to  discover  a  relatively  complete  list  of 
even  the  potential  determinants."^  Finally  it  concluded  that  if  would 
be  difficult  to  make  inferences  from  the  data  since  the  sample  could  not 
be  construed  as  representative. 

SAI .  Beltramo  and  Jordan  performed  a  review  of  dual-source 
competition  studies  for  Headquarters ,  Naval  Material  Command. They 
observed  that  much  of  the  planning  and  cost  estimating  currently  being 
done  for  systems  where  competition  is  contemplated  are  not  based  upon 
sound  economic  theory  or  supported  by  meaningful  empirical  research. 

In  critiquing  the  TASC  methodology,  one  concern  SAI  had  was  that 
TASC  eliminates  early  sole-source  buys  and  thus  projects  its  sole-source 
learning  curve  from  few  data  points.  They  reanalyzed  data  fnon  the 
Sparrow  AIM  7F  guidance  and  control  components  which  account  for  about 
90  percent  of  the  missile's  cost.  Projected  sole-source  learning  curves 
were  estimated  using  three  cases: 

a.  Average  first  unit  cost  for  three  Raytheon  buys  prior  to 
competition  equals  75.3  percent  curve. 

b.  Average  first  unit  cost  first  for  three  Raytheon  buys  prior  to 
competition  (to  show  effect  of  leaving  off  first  buy)  equals  74.1 
percent. 


c.  Cumulative  average  costs  for  four  Raytheon  buys  prior  to 
competition  equals  76.2  percent. 

Beltramo  and  Jordan  calculated  the  effects  of  competition,  as  shown  on 
table  3-3,  for  the  three  cases  as  discussed  earlier  in  the  chapter. 
They  concluded  that,  depending  on  the  case  chosen,  there  was  an  increase 
in  cost  of  from  20.5  to  27.5  percent  over  the  estimated  sole-source 
cost,  including  nonrecurring  costs.  This  is  in  contrast  to  a  16  percent 
projected  savings  by  TASC. 

They  reviewed  three  reports  and  attempted  to  identify  the  source  of 
"misinformation"  about  the  AIM-7F  saving  government  money.  They 
concluded  that  the  TASC  analysis  was  accomplished  through  a  misuse  of 
data.  First,  they  asserted  that  TASC  omitted  the  first  and  last  data 
points  which  "can  introduce  significant  errors  into  projections  and 
naturally  enhances  the  chances  of  finding  shifts  and  rotations.  Tney 
also  noted  the  misuse  of  data  by  TASC  to  prove  a  point  such  as  including 
the  AIM  9-B  Sidewinder  in  the  analysis  when  it  had  only  one  sole-source 
buy  prior  to  competition  and  thus  it  was  impossible  to  establish  sole- 
source  cost.  They  also  pointed  out  that  the  AIM-9D/G,  which  increased 
cost  by  74.5  percent,  was  omitted  by  TASC  an  were  Mark  46  torpedo  and 
Shillelagh  missile,  which  also  increased  government  costs.  Tney 
concluded  by  saying  that  TASC  analysis  has  three  critical  problems  which 
erode  any  confidence  one  might  place  in  its  findings: 
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1)  Only  competitive  procurements  which  apparently  resulted  in 
cost  savings  were  included.  2)  Important  data  which  are 
essential  for  objectively  analyzing  the  effects  of  competition 
were  omitted  from  the  systems  considered.  3)  Although  the 
report  enthusiastically  supports  dual  source  competition  in 
the  procurement  of  weapon  systems,  the  logical  conclusion 
drawn  from  the  analyses  it  provides  does  not  support 
competi t i on . ^ 

LM I  82.  Yet  another  review  of  dual-sourcing  studies  was  accomplished  by 
the  Logistics  Management  Institute  (LMI )  for  the  deputy  under  secretary 
of  defense  for  research  and  engineering  (acquisition  management ) .33  In 
this  study  LMI  made  the  following  conclusion  after  reviewing  the  ECOM, 
IDA,  and  APRO  studies: 

Attempts  to  use  prior  evidence  of  savings  achieved  from 
competitive  reprocurements  to  guide  future  competitive 
decisions  are  only  appropriate  when  such  evidence  is  adjusted 
for  the  characteri st i cs  that  influence  the  level  of  savings 
achieved.  In  most  instances,  absence  of  adequate  data  precludes 
such  an  evaluation.^ 

Sherbrooke  83.  A  study  was  completed  for  Headquarters,  Naval 
Material  Command  by  Sherbrooke  and  Associates  in  1983.-^  Their  work 
critiques  the  ECOM,  APRO,  IDA,  and  TASC  studies  as  well  as  an  OSD 
model.  Their  concern  with  the  ECOM  study  was  similar  to  others  in 
that  there  was  no  inflation  or  rate  adjustments,  nonrecurring  costs  were 
not  considered,  and  learning  was  not  projected.  They  believe  that  the 
IDA  studies  fall  short  for  the  same  reasons. 

They  believe  that  the  APRO  studies  present  a  more  reasoned  approach 
since  estimates  of  parameters  for  first-unit  cost  and  learning-curve 
slope  are  estimated  from  data  and  not  from  regression  equations.  For 
example.  Shillelagh  cost  data  was  used  to  develop  estimates  for 
first-unit,  cost  and  slope,  which  provided  accurate  estimates.  If 
aggregate  data  had  been  used,  according  to  Sherbrooke,  a  much  steeper 
slope  that  would  have  been  obtained  that  would  have  il logically 
reflected  some  nonrecurring  costs.  They  also  felt  that  competition 
savings  were  overstated  as  no  production  rate  was  included  in  the  study. 
Also,  they  did  not  favor  APRO  calculating  savings  percentages  over  total 
program  production  quantity. 

As  with  SAI,  Sherbrooke  researchers  were  disenchanted  with  the  TASC 
model.  They  believe  that  there  is  substantial  reason  for  skepticism, 
noting  the  illogical  conceptual  framework  and  the  less  than  rigorous 
statistical  work  behind  the  parameters  estimates. 

They  criticized  TASC  for  not  publishing  its  basis  for  deflation  and 
pointed  out  that  this  can  create  uncertainty.  For  example,  they  showed 
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that  the  1979  price  used  by  TASC  was  56  percent  of  the  1975  price  for 
tiie  leader  on  the  Sparrow  program  instead  of  the  50  percent  as  estimated 
by  TASC.  When  these  revised  data  are  used  in  a  reanalysis,  the 
procurement  does  not  appear  to  have  produced  savings,  a  conclusion  also 
reached  by  SAI, 

The  primary  Sherbrooke  concern  is  that  TASC  is  trying  to  estimate 
too  many  parameters  from  too  little  data.  Specifically  it  could 
simultaneously  be  trying  to  estimate  as  many  as  five  parameters: 
first-unit  cost,  pre-  and  post-competition  learning  curve  rates, 
production  rate  parameter,  and  optimal  production  rate  from  only  five  to 
seven  years  of  data. 

After  these  general  comments,  Sherbrooke  examines  spec i f i _  missile 
programs  and  identifies  individual  problems.  Reanalysis  of  the  Sparrow 
program,  for  example  showed  the  following: 


TASC  ReA na lysis  t 


F i rst-Uni t  Cost: 

415,336 

426,543 

(135.5) 

Noncompetitive  Cost  Improvement 

Curve  Parameter: 

.846 

.831 

(21.9) 

Competitive  Cost  Improvement 

Curve  Parameter: 

.777 

.824 

(31.5) 

Production  Curve  Parameter: 

.935 

N.S. 

-- 

The  reanalysis  shows  that  the 

competi t i ve 

cost  improvement  parameter 

flattens  out  more  than  that  calculated  by 

TASC  (with  almos 

t  no  change) 

and  that  the  production  rate  parameter  is  not  significant. 

In  contrast 

to  the  TASC  analysis  which  stated  it  could 

not  identify  the 

statistical 

significance  of  its  results,  the 

Sherbrooke 

analysis  was  significant  at 

the  95  percent  level. Similar  results  were  shown 

for  General 

Oynami  cs : 

General  Dynamics 

T_ASC 

Reanalysis 

t_ 

F i rst-Uni t  Cost: 

450,186 

437,171 

(31.9) 

Noncompetitive  Cost  Improvement 

Curve  Parameter 

.874 

.303 

(  3.3) 

Competitive  Cost  Improvement 

Curve  Parameter: 

.759 

.803 

(  5.2) 

Production  Cate  Curve  Parameter: 

.923 

N.S 

- 

Analysis  was  attempted  with  the  Bullpup,  TOW  and  Sidewinder  data.  The 
Bullpup  had  the  same  results  as  did  Sparrow,  and  Sherbrooke  felt  too 
little  data  existed  to  accomplish  TOW  or  Sidewinder  AIM-93  analysis. 

They  recommended  that  the  TASC  approach  be  dropped  entirely,  both 
as  a  research  tool  for  considering  past  programs  and  as  an  ex  ante  model 
supporting  decision  makers. ^  In  its  place,  Sherbrooke  recommended  the 
use  in  ex  ante  analyses  of  learning  curves  and  rate  parameters  that  are 
specific  to  product  groups. 


Cone  In  si ons 

There  is  currently  no  robust  theory  for  identifying  the  projected 
impact  of  introducing  competition  in  the  production  stage  of  the  weapons 
acquisition  process.  The  models  have  become  more  comprehensive  over 
time  as  the  data  have  been  repeatedly  reanalyzed.  Moreover,  the  more 
recent  research  has  attempted  to  reach  beyond  contract  data  and  identify 
explanatory  variables  for  savings  estimates.  Beltramo  considers  the 
method  of  compet i t i on--whether  buy-out  or  split  avard--as  being  that 
variable.  Greer  and  Liao  identify  contractor  "hungri ness"--measured  by 
capacity  utilization  in  the  aerospace  industry--as  being  the  key 
predictor  of  savings.  However,  both  studies  rely  on  estimates  of 
savings  or  loss  that  have  little  statistical  validity  for  the  reasons 
already  delineated. 

An  analytical  model  to  describe  pricing^  by  contractors  dealing 
with  the  government  ought  to  consider  (1)  the  technology  involved  in 
producing  a  given  item,  (2)  the  markets  for  the  firms'  inputs,  and  (3) 
the  business  and  final  product  marketing  climate  in  which  the  firm 
operates.  A  robust  pricing  theory  would  have  to  identify  changes  in 
these  aspects  over  time  to  be  able  to  account  for  changes  in  price  over 
time.  Beltr  imo,  and  Greer  and  Liao  attempt  to  focus  on  the  third  aspect 
without  adjusting  data  over  time  to  account  for  the  first  two  aspects. 

Any  model  that  accurately  describes  the  defense  procurement 
process  over  time  would  have  many  more  variables  arranged  in  a  system  of 
interactive  equations  than  are  normally  included  in  such  a  study. 
Simpler  models  suffer  both  from  omitted  variable  bias  and  from  a  related 
problem-simultaneity  bias.  Even  if  an  accurate,  parsimonious  model  is 
developed  t~  account  for  historical  relationships,  such  models  are  apt 
to  forecast  poorly  whenever  relationships  among  the  conditions  bearing 
on  the  procurement  vary  from  those  prevailing  when  the  data  used  to 
estimate  the  model  were  generated. 

Let  us  rirst  consider  the  paper  by  TASC.  While  it  provides  some 
useful  insights  and  suggests  an  innovative  solution,  tne  author  agrees 
with  other  analysts  that  there  are  flaws  in  the  mo  1  that  seem  to 
counterindi cate  an  uncritical  forecasting  application.  Certainly  the 
■noiel  does  not  perform  as  well  as  the  paper  claims.  We  find 
shortcomings  in  the  general  theory,  in  the  data,  and  in  the  statistical 
method.  And  the  researchers  nave  omitted  from  the  studies  the  details 
necessary  to  evaluate  the  work  more  completely. 

As  previously  discussed,  the  TAS.  research  consists  of  three 
models.  The  first  is  a  basic  learning  curve  model,  or  in  TASC's 
terminology,  a  "cost  improvement  curve"  model.  The  unit-cost  expression 
i s  conv ant i ona 1 : 

Y  =  IV)-'  - 

Ph.ora; 
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Y  =  unit  cost 


Q  =  cumulative  production 

A  =  first-unit  cost 

B  =  parameter 

The  second  model  is  an  extension  of  the  cost-improvement  curve  model 

with  tne  addition  of  a  term  to  capture  the  effects  of  changes  in 
production  rate.  This  provision,  based  on  the  expectation  that  the 

region  of  decreasing  costs  covers  most  defense  industry  production, 
makes  the  TASC  Rate  Analysis  Curve  model: 

Y  =  AQBRC 
Where: 

Y,  Q,  A  =  as  above 
R  =  production  rate 
B ,  C  =  parameters 

Production  rate  measures  are  not  available,  so  a  proxy  measure-- 
annual  lot  size--is  used  in  their  stead. 

These  models  are  estimated  using  a  nonlinear  estimating  technique. 
An  empirical  example  is  made  using  data  from  the  AIM  7  (Sparrow)  missile 
program  as  reported  by  SAI .  An  apparently  innovative  technique  is  used 
to  make  the  regressing  equation  consistent  with  the  data,  which  include 
average  lot  costs.  Unfortunately .  that  estimating  equation  is  not 
presented;  rather,  we  are  left  to  infer  it.  Since  the  estimating 

equation  is  neither  linear  nor  easily  linearized,  non-linear  "curve 

fitting"  techniques  are  used.  Again,  we  are  not  told  what  curve  fitting 
strategy  is  followed,  only  that  it  is  effected  by  means  of  Newton's 

method,  a  procedure  for  finding  approximate  solutions  by  means  of 
successive  iterations.  Further,  there  are  only  eight  observations  in 

the  SAI  data  set. 

Perhaps  the  biggest  problem  with  the  TASC  production  rate  model  is 
its  simplistic  form.  Ironically,  the  study  goes  to  some  length  to 
establish  the  problem  of  omitted  variable  bias  in  the  simple 
cost-improvement  model.  And  our  discussion  of  the  elements  a 
descriptive/  forecasting  model  ought  to  contain  should  serve  to 
illustrate  the  extent  to  which  the  TASC  production  rate  model  is  itself 
subject  to  ommitted  variable  bias. 

The  only  estimation  results  presented  were  parameter  estimates  and 
the  sum  of  squared  error  terms  (SSF1,  a  goodness  of  fit  measure.  Tne 
study  excused  the  absence  of  any  other  statistical  informat'on  with  the 
remark  that  nonlinear  curve-fitting  techniques  lacked  associated  test 
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statistics.  Such  nonlinear  techniques  follow  one  of  two  forms: 
likelihood  function  maximization  or  squared-error  minimization.  TASC 
used  the  squared-error  minimization  technique,  which  can  be  tested  for 
statistical  significance.  Specifically,  the  likelihood  ratio  test  can 
be  used  to  test  whether,  for  example,  the  parameter  in  equation  (2)  is 
different  from  zero,  which  in  effect  tests  if  there  is  any  benefit  to 
adding  the  production  rate  term  to  the  simple  cost-improvement  model 

Comparison  of  the  SSEs  from  the  cost-improvement  model  with  those 
from  the  production  rate  model  reveals  a  curious  fact.  The  addition  of 
a  variable--production  rate--to  the  cost-improvement  model  yields  the 
production  rate  model.  Therefore,  one  would  expect  SSE  to  qo  down  and 
Rseudo-R^  to  go  up,  going  from  equation  (1)  to  equation  (2).^  This  is 
not  the  case;  the  cost  improvement  model  gives  a  higher  Pseudo-R^  than 
the  production  rate  model  (.995156  to  .989044,  our  calculation),  and  the 
cost-improvement  model  also  gives  a  smaller  SSE  (6.293E8  to  6.369E3). 
This  may  indicate  that  the  algorithm  used  to  implement  the  optimization 
routine  is  not  sufficiently  powerful  to  achieve  convergence  on  more 
complex  problems  without  making  search  steps  larger  and  coarsening 
convergence  tolerances.  This  question  is  largely  ignored  in  the  TASC 
reports,  while  much  is  made  of  the  fact  that  the  SSE  for  fitting  total 
lot  cost  is  improved.  The  SSE  for  total  lot  cost  is  not  the  appropriate 
indicator  for  goodness  of  fit;  average  lot  cost  is  the  measure  fitted  by 
the  equation.  We  can  find  the  estimated  total  lot  cost  by  multiplying 
estimated  average  lot  cost  for  a  given  lot  by  the  number  of  units  pro¬ 
duced  in  that  lot.  The  better  fit,  measured  by  SSE,  of  total  lot  cost 
is  a  coincidence,  due  to  the  fact  that  the  larger  model  fits  average 
costs  better  for  the  later,  larger  lots.  This  is  to  be  expected,  since 
production  has  become  more  routine  by  the  time  the  later  lots  are 
produced. 

The  third  model  presented  in  the  TASC  paper  incorporates  into  the 
production  rate  model  consideration  of  whether  or  not  a  given  lot  is 
procured  under  dual -sou rci ng.  The  technique  used  is  to  incorporate  a 
dummy  variable  to  "shift  and  rotate"  the  cost-improvement  curve.  The 
sped  f  l  c  form  is: 

V  =  a  Texp ( d *0C  )  ]Q (6-t-fDC  )qc 

where  DC  is  a  binary  with  a  value  of  one  when  the  lot  is  procured  under 
competition  and  zero  otherwise.  From  a  curve-fitting  perspective,  this 
is  a  reasonably  artful  use  of  dummy  variables. 

This  extended  model  is  then  fitted  using  both  the  SA1  data  and  data 
s j u h> i  lea  by  the  AIM-7  program  office.  These  latter  data  are  described 
in  the  report  as  more  reliable  than  those  published  by  SAI.  The 
question  naturally  arises  as  to  why  the  SAI  data  were  used  in  the  first 
place.  Why  weren't  all  the  models  fitted  using  the  superior  data? 
Farther,  the  program  office  data  values  average  about  5U  percent  smaller 
than  the  corresponding  values  in  the  SAI  set.  Clearly,  some  different 
variable  definition  is  involved,  but.  there  is  no  discussion  of  this  fact 
or  what  lies  behind  it. 
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The  first  four  production  lots  of  the  AIM-7  were  produced  under 
sole  source  arrangements,  and  then  were  procured  in  1977-1979  (1939  in 
the  SAI  set)  under  the  split-buy  arrangement.  In  effect,  the  shift  and 
rotation  parameters  are  fitted  to  only  three  observations,  using  the 
program  office  data.  Since  overall  the  model  is  fitted  from  trended 
data  in  five  total  parameters  for  seven  observations,  it  is  not 
surprising  that  the  fit  is  quite  good  and  that  the  last  three 
observations  are  estimated  with  almost  no  error.  This  virtually  er.sjres 
a  small  SSE  in  total  lot  cost.  In  addition,  the  very  small  data  set 
makes  the  strong  claims  TASC  advances  for  this  model  appear 
exaggerated. 

Finally,  the  construct  of  the  "best  competitive  curve"  in  chapter  4 
of  the  paper  seems  questionable.  The  model  specifies  that  the  original 
source's  learning  curve  both  shifts  and  rotates  with  the  introduction  of 
competition.  As  drawn  in  figure  3-2  on  page  62,  the  post-competition 
portion  of  the  original  source's  learning  curve  is  steeper  than  the 
"best  competitive  curve."  The  accompanying  description  suggests  that 
this  must  always  be  so.  According  to  the  model,  if  competition  were 
introduced  from  the  onset,  it  would  both  shift  and  rotate  the  learning 
curve  downward  to  a  locus  everywhere  below  the  post  competition  learning 
curve  shown  in  the  figure.  We  doubt  the  usefulness  of  the  latter  as  a 
predictive  tool  as  it  might  not  be  possible  for  the  sole-source 
contractor  to  shift  his  costs  to  always  intersect  with  the  best 
competitive  curve. 

Greer  and  Liao  offer  an  alternative  approach,  incorporating  into 
their  model  consideration  of  an  important  measure  of  business 
conditions:  capacity  utilization.  Basically,  they  argue  that  an 

important  element  in  forecasting  a  contractor's  pricing  behavior  is  the 
percentage  of  his  total  productive  capacity  that  is  being  utilized. 
When  capacity  utilization  is  low,  firms  are  "hungry"  and  likely  to 
respond  to  competitive  pressure  with  lower  bids  than  otherwise.  Since 
capacity  utilization  rate  data  are  not  available  for  individual  firms, 
Greer  and  Liao  use  the  average  rate  of  capacity  utilization  for  the 
appropriate  sector  of  industry  as  a  proxy.  While  capacity  utilization 
is  an  important  consideration,  there  are  both  statistical  and  theoretic 
shortcomings  in  their  paper. 

As  part  of  the  tangential  argument  to  support  the  contention  that 
excessive  zeal  in  government  cost-minimization  is  misplaced,  Greer  and 
Liao  attempt  to  show  that  defense  business  is  characterized  both  by 
lower  returns  on  investment  and  higher  risks  than  is  civilian  business. 
Their  approach  is  to  regress  by  ordinary  least  squares  (JLSl  two 
measures  of  return  on  i nvestment--prof i t  on  sales,  and  profit  on  net 
worth--agai nst  the  percentage  of  a  firm's  business  accounted  for  by 
the  government.  (They  use  government  business  as  a  proxy  for  DOD 
business.)  Their  data  base  is  substantial,  covering  some  26  finis 
in  the  aerospace  industry  for  the  20-year  period  from  1963  to  1932. 
They  run  separate  regressions  for  earh  firm. 


The  regression  of  profit  measures  against  the  portion  of  total 
business  done  with  the  government  indicates  a  negative  relationship. 
Possible  errors  enter  the  analysis  when  an  attempt  is  made  to  analyze 
risk  using  this  model.  The  authors  project  the  prof i tabi 1 i ty  that  each 
firm  could  expect  both  at  0  percent  government  business  and  at  100 
percent  government  business.  They  then  choose  as  measures  of  riskiness 
the  standard  errors  of  prof itabi 1 ity  at  no  government  business  and  they 
compare  the  two  to  assess  the  relative  riskiness  of  doing  business  with 
the  government. 

We  have  three  criticisms  of  this  procedure.  First,  the  linear 
extrapolation  of  the  extremes  of  business  composition  for  firms  whose 
average  concentrat ions  of  business  run  in  the  range  of  40-60  percent 
(according  to  the  authors)  seems  likely  to  be  inaccurate.  Second, 
standard  deviation  is  an  appropriate  measure  of  risk  only  under  special 
circumstances.  For  the  analysis  at  hand,  however,  it  may  be  the  best 
simple  measure  available.  The  most  serious  problem  concerns  the  measure 
of  risk  used. 

When  projecting  the  profitability  (P)  associated  with  a  given 
proportion  of  government  business  (GB),  using  the  following  model: 

?  -  a  +  b*GB  *■  e 

where  e  is  a  residual,  the  variance  of  a  forecast  for  some  Pq  depends 
on,  among  other  things,  the  difference  between  the  associated  GBq  and 
the  mean  of  GB  in  the  sample  used  to  fit  the  regression.  Forecasting 
far  from  the  sample  mean,  as  is  the  case  here,  implies  a  fair  amount  of 
inaccuracy.  This  procedure  will  bias  the  calculation  of  profit 
variability  at  business  extremes  unless  the  sample  mean  for  each  firm 
is  50  percent  government  business.  We  can  also  state  the  direction  of 
this  bias.  If  the  average  portion  of  government  business  is  less  than 
50  percent,  then  the  bias  will  tend  to  make  government  business  appear 
more  risky  than  private  sector  business.  Conversely,  if  government 
business  is  greater  than  50  percent  of  the  total,  the  bias  will  tend  to 
make  private  business  look  more  risky  than  government  business.  Since 
the  authors  do  not  provide  either  firm  or  total  sample  business 
composition  means,  we  cannot  make  a  specific  assessment  of  the  effects 
on  the  bias  in  this  case. 

The  main  thrust  of  the  paper  involves  estimating  two  alternative 
pricing  models  and  using  the  results  to  argue  that  success  in  dual- 
sourcing  (measured  in  terms  of  program  savings)  depends  on  the  amount  of 
excess  capacity  among  affected  contractors.  Wnile  this  is  plausible, 
tnere  are  some  problems  with  the  statistical  wort-  and,  in  particular, 
witn  errors  in  interpreting  the  re>,  jits. 

Greer  and  Liao  posit  the  two  models  that  have  already  been 
d '  scussed.  There  is  a  production  rate  mode  I  s  i  ill  i  I  ir  to  that  developed 
in  the  TASC  paper,  written  as  follows: 
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and  a  capacity-utilization  model,  written: 

P  =  aQbllcexp(dM  +  f  N ) 

Where: 

P  =  unit  price 

Q  =  cumulative  production 

R  =  production  rate 

U  =  percentage  of  plant  capacity  utilized  in  the  industry 

M  =  a  dummy  variable  of  the  value  of  one  if  the  buy  was  under 
split-buy  and  zero  otherwise. 

N  =  dummy  variable  taking  on  a  value  of  one  if  the  buy  was  in 
the  first  year  after  a  winner-take-all  competition  and  zero 
otherwi se. 

Estimating  equation  (5)  separately  on  the  data  for  each 
gave  rather  poor  results,  both  for  the  original  system  developed  and  for 
the  second  source.  However,  the  authors'  statistical  "iethods  cojH 
perhaps  be  improved  upon.  We  will  cover  these  matters  below  in  our 
discussion  of  their  treatment  of  the  capacity-utilization  model. 

The  capacity-utilization  model  gave  results  that  on  the  surface 
appear  reasonable,  both  for  the  original  developing  firm  and  for  tne 
second  source.  And  this  model  form  gave  better  predictive  results  tnan 
did  the  production  rate  model.  However,  tnere  are  some  statistical 
issues  which  are  troublesome,  though  not  necessarily  fatal  to  the 
argument.  First,  the  authors  do  not  provide  anv  of  the  conventions' 
summary  statistics  for  any  of  their  regression  analyses.  Since  there 
are  many  of  these,  this  material  could  bp  given  in  an  appendix. 

The  paper  does  provide  the  average  and  tne  median  parameter  values 
from  separate  regressions  on  each  project.  Tne  mean  and  median  are 
calculated  after  eliminating  "outliers."  It  is  poor  procedure  to 
eliminate  an  outlier.  Tne  definition  of  an  outlier  is  based  on  s na 
notion  of  what  are  reasonable  results  coupled  with  the  observed  centra' 
tendency  of  all  results.  The  exceptional  case  often  carries  r,rS 
information  than  the  routine--and  may  be  an  indication  of  the  inadeiiacy 
of  theory,  for  example.  The  bottom  line  on  elrvnating  "outliers"  is 
that  it  leads  to  a  tendency  to  discard  information  at  variance  witn  our 
preconceived  notions. 

There  are  also  more  efficient  ways  to  find  "average"  parameter 
values.  A  common  practice  is  to  pool  the  data  from  all  activities  :ne 
believes  can  be  described  by  a  single  model,  and  estimate  a  parameter 
set  jointly. Tnis  gives  more  efficient  (accurate',  estimation  af  tne 
structural  parameters  along  with  <a  procedure  test  whether  or  not  an 
"average"  set  of  parameters  des'nring  these  activities  'S  apD^'j-iate. 


Finally,  some  of  the  conclusions  argued  by  Greer  and  Liao, 
apparently  based  on  the  stated  results  from  their  capaci ty-ut i 1 i zat i on 
model,  cannot  be  substained  based  on  those  results.  Let's  consider 
their  fit  of  the  capacity-utilization  equation  using  median  parameter 
values: 

b  *  -.278  d  =  -.201 

c  =  1.250  f  =  -.854 

On  page  4-4  of  their  paper,  the  authors  argue  that  competition  's 
necessary  to  induce  a  firm  to  lower  its  price  when  it  has  unused 
capacity.  On  page  4-13,  they  go  on  to  say,  "There  is  no  reason  to 
believe  prices  react  to  'hungriness'  when  the  acquisition  program  is 
conducted  without  competition."  The  model  results  do  not  bear  out  these 
assertions. 

Let  us  consider  two  commonly  used  measures  of  responsiveness:  the 
partial  derivative  of  price  with  respect  to  capacity  utilization,  and 
the  elasticity  of  price  with  respect  to  capacity  utilization.  While  the 

first  measure  is  a  common  mathematical  term,  the  second  should  be 

defined.  The  elasticity  of  price  with  respect  to  capacity  utilization, 
or  the  percentage  change  in  price  accompanying  a  one  percent  change  in 
capacity  uti 1 ization,  is: 

£  _ _  Percentage  change  in  price 

Percentage  change  in  capacity  utilization 

An  equivalent  definition  is 

r  -  >1 

r-  “  ;,U  P 

If  we  take  the  partial  derivative  of  price  with  respect  to  capacity 
utilization,  we  find  it  to  be 

_'_P  =  c ( aQ^t)  ( c" L '  exp(d*m  +  f*n) 

“IT 


Tnis  implies  that  for  any  initial  capaci ty-ut i 1 i zat i on  level,  and  any 
value  of  c,  the  higher  the  P  the  greater  will  be  the  change  in  price 
that  accompanies  a  small,  fixed  change  in  capacity  utilization.  Since 
both  d  and  f  are  negative,  P  will  be  lower  in  the  presence  of 
competition  than  in  its  absence.  Hence,  we  conclude  from  the  model  that 
toe  change  in  price  generated  by  a  fixed  change  in  capacity  utilization 
is  greater  in  the  absence  of  competition  than  in  its  presence. 

In  the  model,  the  elasticity  of  price  with  respect  to  capacity 
utilization  levels  is  invariant  wi‘h  respect  to  whether  or  not  the  buy 
is  competed.  The  elasticity  is: 


r  ci  P  U  -  P  U  - 
UP  UP 

And  c  is  a  constant.  Both  these  results  show  that  the  structure  >f  vie 
capacity-utilization  model  is  not  consonant  with  the  authors'  remar*'.  s 
about  competition  being  a  necessary  condition  to  extract  price 
concessions  from  contractors  having  excess  capacity. 

A  numerical  illustration  may  be  useful.  Suppose  a  =  1.0  (no  value 
is  given,  but  any  positive  number  will  do  for  the  example).  Further 
suppose  that  Q  =  1000  and  U  =  Ob  percent.  With  no  competition,  ?  = 
27.05.  A  fall  in  capacity  utilization  to  60  percent  yields  a  price  of  P 
=  24.47,  for  a  price  reduction  of  2.50.  The  arc  elasticity  of  price 
with  respect  to  capacity  utilization  is  5  =  1.25.  If  we  alternat i vel y 
consider  a  split-bid  competitive  buy,  with  IJ  =  65  percent  capacity 
utilization  and  cumulative  production  of  Q  -  1000,  then  P  =  22.12;  and  a 
fall  in  capacity  utilization  to  U  =  60  percent  gives  P  =  20.01,  fcr  a 
price  reduction  of  only  2.11.  Again,  the  arc  elasticity  of  price  with 
respect  to  capacity  utilization  is  E  =  1.25. 

Greer  and  Liao  also  contend  (p.  5.1)  that  savings  or  losses  in 
dual -sou rci ng  can  largely  be  explained  by  capacity  utilization,  with 
losses  from  dua 1 -sourc i ng  occurring  at  high  capacity-utilization  levels. 
Their  model  contradicts  this  proposition  as  well,  as  can  be  see'1  w'fh 
another  numerical  i  1  lustration.  Suppose  again  that  a  -  1.0,  j  =  1 , -J-) j , 
but  U  =  30  percent.  Now  suppose  a  spl it- buy  competition  is  initiated. 
The  original  source's  price  before  competition  i  s  3  =  3d. 06.  ' 

post-competition  price  is  P  -  28.63  for  a  unit  price  improvement  of 

6.33.  Suppose  that  all  other  figures  remain  the  same,  but  that  toe 

capacity  utilization  level  is  U  =  60  percent.  The  noncomoetiti ve  prize 
is  now  P  =  24.47.  After  competition  is  introduced,  the  price  falls  f 
20.02,  for  an  improvement  of  only  4.45.  Since  in  the  model  foe 
introduction  of  competition  improves  price  more  at  higher  levels  of 

capacity  utilization,  we  contend  tnat  the  model  contradicts  ft  he'-  fan 
sjpports  the  authors'  argument. 

Greer  and  Liao's  argument  has  intuitive  appeal,  but  toe  move 
discussion  points  to  an  i  1  I -dps  i  gned  model.  It  could  be  impo,>»-  1 1 

give  more  sensible  responsiveness  results  ny  ma-mg  the  exponent  on 
capacity  utilization  depend  on  the  presence  or  jr.sence  of  compe*  ' V  on , 
and  by  making  coefficients  on  the  dummies  for  the  i  of  o  1  mt  i  m  ;f 

competition  depend  on  capac i t y-ut i 1 i zat i on  levels,  as  follows: 

(6a)  P  -  i  c>c  '  Vo"  a  'expMVd'  N  •  .-‘X**'  f-J 


I’lf  1  ry 

In’;  chapter  •  ipened  with  »  f  >  i  c  » s  •:  i  on  nr  prli.";  •  ■  -  .  • 

t*.  tempted  to  guant  i  fy  the  cos'  ml  benef  its  of  :,p-; 

reviewed  tnei  -  data,  methods ,  md  mi  ior  co*v  1  is’  )':•».  (t  fm.o  ;  •  m 
d’fferent  studies  analyzing  t  same  p mgr  mi  ved 
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estimates  of  savings  or  increased  cost  to  the  government.  It  also  found 
that  no  rul e-of-thumh  exists  that  points  to  a  fixed  percentage  savings-- 
such  as  25  percent--accrui ng  to  the  government  whenever  a  weapon  system 
is  competed  in  production. 

We  identified  several  reasons  why  estimates  of  savings  were 
different  between  studios: 

(11  Different  inflation  adjustments  and  data  manipulation 

(2)  Different  bases  for  calculation  of  savings  estimates,  such  as 
first  calculation  of  savings  competitive  buy  versus  total  program 

(31  Inclusion  of  different  systems 

''4!  Specification  bias  since  some  calculations  net  out 
nonrecurring  costs  while  others  report  gross  savings 

None  of  the  studies  considered  extra  costs  to  the  government  such 
as  source-selection,  contract  admi ni st rat i on  or  configuration  management 
costs.  Moreover,  no  study  addressed  the  potentially  favorable  or 
unfavorable  impacts  on  other  stagps  of  the  program  such  as  better  spare 
parts  pricing  because  of  the  existence  of  a  technical  data  package,  or 
claims  resulting  from  a  poor  technical  data  package.  And  the  majority 
of  systems  studied  to  date  have  been  low  dollar  value,  high  quantity 
systems  that  were  mass  produced.  To  extrapolate  the  results  in  a 
predictive  sense  to  high  dollar  value  low  quantity  systems  should  be 
done  very  cautiously. 

Published  critigues  of  these  studies  were  reviewed.  They  were 
unanimous  in  their  conclusions  that  there  is  not  adequate  data,  even  if 
there  were  a  methodology,  to  develop  in  a  predictive  sense  a  statistical 
procedure  to  capture  the  simultaneous  equation  system  necessary  to 
Hentify  the  impacts  we  discussed  ear  1 ier--factor  inputs,  markets,  and 
technology.  (This  conclusion  was  initially  stated  in  the  19/2  ECOM 
study!'  In  fact,  we  showed  how  at  least  one  study  yields  results  that 
contradict  the  assertions  claimed  by  the  analysts. 

Notwithstanding  the  shortcomings  discussed  above,  three  points  must 
be  made.  Dual -sourci ng  of  weapons  is  done  for  many  purposes--to  develop 
s  irge  capability,  to  advance  technological  progress,  and  to  foster 

contractor  cooperation  among  others--nof  just  to  reduce  weapon  systems 
cost.  Therefore,  a  sample  of  programs  that  have  been  competed  cannot 
be  considered  a  random  sample  and  representative  of  all  procurements 
s  i o h  that  the  results  can  be  extrapol ated  to  other  programs ,  something 
’’.bat  Oas  been  done  at  the  highest  levels  ot  the  execut  ive  and 
legislative  branches.  And  a  second  point  which  stems  from  the  first  is 
t"  at  we  do  not  X  now  when  compet  it  ion  leads  to  reduced  cost  or  prof  i *  . 
It  the  government  is  using  its  leverage  in  the  short,  run  to  reduce 

prjfir,  especially  in  1 i ght  of  »vi  fence  that  bin’s  profits  on  defense 

o  jS’oess  are  legs  than  comme  rc  i  a  '  business,  we  may  be  her.ef  ■  tt  i  ng  i  n  the 

s-grt  c  jn  at  toe  exppnse  of  the  lang-r  jn  vi  ih'li’y  of  our  defense 


industry.  The  available  data  that  we  have  at  the  contract  price  level 
(cost  plus  profit)  simply  does  not  allow  us  to  draw  these  important 
distinctions.  Finally,  we  must  avoid  uncritical  application  of 
mechanical  techniques  to  forecast  future  savings  from  historical  data. 

Chapter  4  will  explore  program-speci f ic  analyses  that  have  teen 
conducted  to  help  determine  the  cost  effectiveness  of  dual -sourci ng. 
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CHAPTER  4 


PROGRAM  SPECIFIC  COMPETITION  ANALYSES 

The  weapons  acquisition  process  requires  that  cost  estimates  he 
conducted  for  a  variety  of  reasons  and  at  a  number  of  levels.  Program 
managers  must  develop  alternative  acquisition  strategies  and  cost 
estimates  to  evaluate  each  strategy's  effectiveness  in  completing  the 
acquisii  on  as  set  out  in  the  program  manager's  program  management  plan. 
In  accordance  with  000  directives,  separate  cost  analyses  must  be 
conducted  for  major  weapons  systems  as  an  independent  test  of  the 
reasonableness  of  program  office  estimates. *  These  analyses  can  be  as 
extensive  as  a  detailed  independent  cost  analysis  (ICA)  in  support  of  a 
Defense  System  Acquisition  Review  Council  (DSARC)  decision  to  approve 
progression  of  a  weapon  system  from  one  acquisition  stage  to  another  as 
discussed  in  chapter  2.  Somewhat  less  detailed  is  an  independent 
sufficiency  review  that  consists  of  a  summary  review  of  a  program  office 
estimate.  Finally,  the  least  detailed  cost  analysis  consists  of  an 
independent  cost  study  that  reviews  program  office  ground  rules  and 
estimating  methodologies.  Whereas  the  first  two  are  compared  to  program 
office  estimates,  the  latter  does  not  necessarily  include  an  evaluation 
and  a  detailed  comparison  with  a  program  office  estimate. 

The  balance  of  this  chapter  outlines  selected  Army,  Navy,  and  Air 
Force  program  office  and  ICA  analysis  methodologies  for  estimating  the 
costs  and  benefits  of  dual -sourcing  weapons  in  production.  First, 
however,  is  a  quick  review  of  cost-estimating  techniques  to  set  the 
stage  for  the  remainder  of  the  chapter. 


Cost  -dist  ima  ti  ng  _M_e_t_hods 

Cost-estimating  methods  used  in  making  an  individual  analysis  v-rv 
widely  among  analysts.  However,  three  methods  of  estimating  are 

generally  available:  analogy,  parametric,  and  the  industrial  engineer¬ 
ing  approaches. 

Analogy  and  parametric  methods  are  "top  down"  approaches  because  they 
forecast  total  program  costs  as  a  single  number.  Analogy  estimates  are 
based  on  data  from  similar  past  programs  or  tasks,  with  adjustments  male 
for  differences  in  the  present  program.  They  generally  involve  the  use 
of  learning  curves  and  their  power  function  adjustments. 

Parametric  estimates  develop  a  cost-estimating  relationship  where 
cost  is  determined  from  an  estimate  of  a  physical  or  performance 

characteristic  of  the  program,  such  as  weight  or  speed  of  the  system.  It 
can  also  be  based  on  the  number  of  personnel  assigned  to  a  level-of- 
effort  task.  One  example  of  a  parametric  cost-estimating  method  was 

developed  by  RCA.  Its  Programmed  Review  of  Information  for  Costing  and 

Evaluation  (PRICE)  model  was  t-v  loped  f )r  estimating  engineering 
development  costs  of  electronic  equipment. 
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The  industrial  engineering  approach  is  a  "bottom's  up"  estimating 
technique  that  estimates  each  of  the  cost  elements  of  the  program.  It 
is  generally  the  most  accurate  as  it  is  accomplished  after  firm  designs 
are  known  an  !  r  iduof ' o  1  technologies  established.  This  technique  can 
also  use  learning  curves  for  individual  labor  elements. 

Although  these  methods  are  discussed  as  discrete  methods,  they 
are  not  used  independently.  The  engineering  approach,  for  example,  can 
itilize  analogy  or  cost  estimating  relationship-,  i  >  e-.tinjte  the 
individual  cost  elements. 


Prog  ram  _A  na  lyses 

The  military  services  and  OSD  all  have  conducted  program  specific 
competition  analyses.  We  will  now  examine  selected  program  specific 
studies,  reviewing  the  factors  considered  in  the  analysis  (as  identified 
in  chapter  2),  methods  of  analysis,  and  statistical  treatment  of  data. 


1 IR  Maverick 

The  HR  Maverick  is  an  imaging  infrared  guided  missile  designed  to 
target  tanks,  bunkers  and  other  ground  targets.  The  missile  was 
designed  by  Hughes  Aircraft  Company  and  has  two  major  secti ons--the 
imaging  infrared  guidance  and  control  section  (GCS)  and  the  center/aft 
section  (CASK  The  GCS  is  unique  to  this  system,  while  the  CAS  is 
common  to  TV,  laser  and  1 1 R  Mavericks. 

Production  of  the  HR  Maverick  began  in  FY  82  with  200  missiles. 
Current  planning  includes  the  production  of  500  units  in  FY  85  with 
competition  starting  in  FY  86  for  the  remainder  of  the  60,664  missile 
requirement  between  FY  82  and  FY  90. 

On  24  November  1982  Headquarters  USAF  directed  that  an  independent 
cost  analysis  of  the  program  be  performed.  The  I C A  was  completed  in 
February  1983  as  an  independent  test  of  the  reasonableness  of  the 
program  office  estimate.^  It  was  developed  using  three  scenarios: 
(a)  a  50/50  split  in  years  1986  through  1990,  (b)  a  50/50  split  in  1986 
with  a  buy  out  from  one  producer  for  the  1987  and  future  requ i rements , 
and  (c)  an  80/20  split  between  Hughes  and  the  second  source. 

The  general  methodology  was  to  make  a  building  block  estimate  of 
the  cost  of  a  Hughes'  sole-source  buy  and  reduce  the  relevant  costs  to 
reflect  the  effects  of  competition.  Methods  used  to  develop  the  Hughes 
estimate  included  the  RCA  Price-H  model,  an  analogous  estimate  of  the  FMS 
program  and  HR  production  lot,  purchase  order  data,  and  the  TV  Maverick 
cost  history  among  other  sources. 

The  methodology  for  estimating  the  effects  of  competition  was  based 
on  the  model  developed  by  The  Analytic  Sciences  Corporation  (TASC),  which 
snowed  three  effects: 


fee  reduction 

other  overall  cost  reduction 
greater  learning 

The  ICA  accepted  the  TASC  numbers  and  took  a  5  percent  fee  reduction,  a 
9  percent  other  overall  cost  reduction,  and  a  4  percent  greater  learning 
factor  against  the  guidance  and  control  section  starting  in  FY  86.  The 
most  optimistic  estimate  in  the  uncertainty  band  estimated  by  TASC  was 
used.  For  the  center  aft  section  only  a  6  percent  fee  reduction  was 
taken  because  the  component  was  presently  being  procured  under  the  TV 
Maverick  program  and  it  was  felt  that  the  learning  had  terminated. 

The  following  results  were  obtained  showing  the  forecasted 
sole-source  cost  and  expected  cost  of  the  three  competitive  scenarios  in 
base  and  then-year  (TY)  dollars: 


TY  Dollars  FY 

M i 1 1 i ons 

82  Dol lars 

50/50  split 

5,794 

4,003 

50/50  FY  86  followed 
by  buy-out 

5,461 

3,794 

80/20  split 

5,651 

3,911 

Sole-source 

5,866 

4,042 

These  figures  show  that  if  the  FY  86  60/60  award  is  made  and  is 
followed  by  a  multiyear  buy-out,  there  would  be  a  savings  of  243  million 
dollars  over  the  projected  sole-source  cost.  This  equates  to  6  percent 
of  total  program  cost  in  FY  82  dollars.  As  stated,  point  estimates  of 
the  probable  cost  under  the  three  scenarios  were  made  by  applying  the 
most  optimistic  average  percentage  reductions  to  the  respective  cost 
elements  estimated  in  the  sole-source  case.  If  the  ICA  had  used  other 
than  the  most  optimistic  estimates,  the  lowest  price  ICA  scenario  would 
have  been  higher  and  much  closer  to  the  SPO  estimate  of  5,513  million 
dol lars. 

In  developing  its  sole-source  estimate,  the  ICA  team  assumed  the 
second  source  would  be  developed  and  "that  after  the  FY  85  buy  of  500 
units  from  the  second  source,  the  Air  Force  decides  to  abandon  the 
notion  of  competition  (for  some  unknown  reason)  and  purchases  the 
remaining  missiles  from  Hughes.  In  addition  to  the  learning  curve 
penalties  of  having  the  second  source  produce  the  500  missiles  on  its 
learning  curve  rather  than  Hughes  produce  them  down  its  learning  curve, 
the  following  second-source  costs  were  also  included  in  the  sole-source 
estimate  because  of  this  assumption:1^ 


4  percent 
8  percent 


nt 

nt  12%  + 


2  % 


4  percent 


TY  $ ,  Mi  1 1 i ons 

tooling 

47.35 

qual i f i cat i on 

87.84 

system  engineering/program  mgt 

36.60 

TOTAL 

171.79 

It  would  have  been  more  appropriate  for  the  I C A  team  to  have  excluded 
these  second-source  development  costs  from  the  sole-source  estimate  if 


it  was  attempting  to  identify  the 
competition  as  stated  in  its  report. 


cost-reduction 

5 


implications  of  the 


It  did  not  use  discounted  cash  flows,  nor  did  it  consider  the 
additional  source  selection  and  contract  administration  costs.  Finally, 
it  discussed  the  impacts  on  program  costs  for  spares  but  did  not 
quantify  them.  Nor  did  it  consider  the  effects  on  the  price  of  other 
government  programs  such  as  the  TV,  FMS,  laser,  and  Navy  I IR  Maverick 
programs . 


LANTIRN 

The  Air  Force  is  presently  developing  the  low-altitude  infrared 
navigation  and  targeting  system  (LANTIRN)  developed  by  Martin-Marietta 
Corporation  (MMC).  The  system  consists  of  two  externally  mounted  pods 
with  720  pod  sets  programmed  for  purchase  and  use  on  the  F - 1 6  and  A-10 
aircraft  for  night  under-the-weather  attack  capability. 

Congress  prohibited  the  obligation  or  expenditure  of  any  FY  83 
development  funds  for  the  targeting  pod  until  there  was  a  competitive 
demonstration  between  the  LANTIRN  targeting  pod  and  the  product  improved 
F  /A - 1 8  FLIR  targeting  pod  developed  by  Ford  Aerospace.  TASC  published  a 
report  in  May  1983  at  the  request  of  the  LANTIRN  program  office  which 
outlined  possible  competitive  strategies  that  would  satisfy 
congressional  and  Air  Force  objectives. ^ 

The  May  1983  submission  updated  a  July  1982  plan  which  initially 
developed  alternative  acquisition  strategies  and  identified  potential  R 
\  0  and  procurement  costs  for  LANTIRN.  The  updated  report  di «s <~m ssed  the 
competitive  fly-off,  five  leader/follower  alternati ves,  and  <  ..opponent 
breakout  program. 

Recurring  cost-savings  estimates  for  the  competitive  fly-off  were 
estimated  at  7  percent  in  then-year  dollars.  This  was  considered  a 
reasonable  estimate  based  on  a  reference  to  the  APRO  78  study  discussed 
earlier.  From  this  figure  TASC  analysts  subtracted  additional 
development  and  schedule  impact  (inflation)  costs,  which  resulted  in  an 
added  cost  of  133.5  million  in  then-year  dollars  for  this  option. 
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1 


The  five  leader/follower  alternatives  included  combinations  us’ng 
the  pods  as  a  set  or  competitive  procurement  of  only  the  targeting  pod. 
Alternate  I  is  the  targeting  pod  only  with  a  production  rate  of  16  per 
month  and  competition  starting  in  lot  2  with  a  multiyear  buy-out  for-  the 
last  two  buys.  Alternative  1 1  a  is  similar  to  1  except  competition 
starts  at  lot  3.  Alternative  lib  is  similar  to  1 1  a  except  production  is 
12  per  month.  Alternative  III(al  involves  competition  for  both  pods 
with  production  at  16  pods  per  month  and  competition  starting  with  lot 
3.  Alternative  111(b)  is  similar  to  Ilia  except  for  production  of  12 
pods  per  month. 

The  analysts  estimated  program  costs  of  the  leader/follower 
acquisition  strategies  by  using  TASC's  production  cost  anal/sis 
predictive  methodology  (PCAM)  discussed  in  chapter  3.  Use  of  this  model 
involves  estimating  the  syscem  developer’s  shift  and  rotation  of  the 
learning  curve.  In  a  predictive  sense,  the  model  assumes  that  tne 
system  developer  alters  his  cost  behavior  just  enough  to  stay  ahead  of 
the  second  producer  in  a  competitive  environment.  This  translates  into 
a  shift  and  rotation  of  the  system  developer's  cost-improvement  curve 
adequate  to  intersect  the  best  competitive  curve  at  the  end  of  the 
planned  production  run.  Furthermore,  once  parity  is  achieved,  if 
production  quantities  are  extended,  the  producer  continues  to  follow  the 
best  competitive  curve.  The  model  calculates  the  system  developer's 
shift  and  rotation  on  the  assumption  that  he  bids  to  win.  And  it 
implicitly  assumes  that  he  has  perfect  insight  into  the  second  source's 
cost  behavior.  Ic  also  assumes  the  second  source  is  always  less 
expensive  than  the  first  source. 

The  calculation  of  a  projected  shift  for  the  system  developer 
involves  projecting  each  producer's  unit  cost  up  to  the  point  of 
competition.  The  difference  between  the  projections  establishes  the 
amount  the  original  producer  must  reduce  his  cost  to  win  tne 
competition.  The  general  equations  required  to  calculate  the  required 
shift  and  rotation  are  presented  below.  They  assume  that  the  developer 
produced  N  units  prior  to  competition  and  that  the  second  source 
produced  K  units  prior  to  competition. 


Cost  of  Developer  N  +  1  unit  = 

A (N  +  1)8  (Y)C 

Cost  of  the  Second  Source  K  +  1  unit  = 

0 (K  +  1)E  (X)F 

Developer's  Cost  Improvement  Curve  Shift  = 
0  (K  +  1)E  1X)F 

A (N )  t  1)3  (  i  )C 


p:’ 


1 


Where: 


A  and  0  represent  first-unit  costs. 

B  and  E  represent  the  log  of  cost  improvement  rate/log  of  2. 

Y  and  X  represent  optimal  lot  sizes. 

C  and  F  represent  log  of  production  rate  parameter/log  of  2  for 
the  developer  and  second  source,  respectively. 

Calculation  of  a  rotation  of  the  system  developer's  cost- 
improvement  curve  is  based  upon  the  cost  reduction  he  must  attain  to 
continue  to  win  the  competitive  awards  through  the  remaining  production 
run.  The  general  form  of  the  equation  used  to  calculate  rotations  for 
the  LANTIRN  system  developer  and  leader  is  logarithmic: 

(In  2)  In  1 /SMa 
In  R  = - 

In  (N  +  1)/M 


Where: 

R  is  the  rotation. 

S  is  the  ratio  of  second  source's  K  +  l  unit  cost/developer's  N+l 
unit  cost. 

M  is  the  number  of  units  produced  when  the  rotated  curve  meets  the 
best  competitive  curve. 

N+l  is  the  first  competitive  unit. 

a  is  the  negative  logarithm  of  best  competitive  rate/developer's 
rate  divided  by  the  In  of  2. 

Using  this  predictive  methodology,  TASC  estimated  the  costs  and  benefits 
in  then-year  dollars  as  reflected  in  table  4.1.^  No  further  detail 
beyond  this  chart  was  reported  in  the  analysis. 

The  TASC  analysts  concluded  that  total  program  cost  reductions 
ranged  between  $118.9  and  $275.2  million  in  then-year  dollars.  They 
included  the  following  caveat  with  the  estimate: 

This  update  of  the  LANTIRN  Competitive  Procurement  Plan 
was  a  quick  reaction  re-look  at  several  acquisition  strategy 
alternatives  and  the  results  are  Rough  Order  of  Magnitude.  We 
recommend  that  if  the  Program  Office  is  interested  in  pursuing 
any  of  these  options,  more  analysis  should  be  done.  In  other 
words,  a  thorough  analysis  of  what  makes  sense  as  a  LANTIRN 
Acquisition  Strategy--a  detailed  examination  of  system 
elements,  contract  options,  data  rights,  etc.,  and  a  carefully 
constructed  cost/benefits  analysis  must  be  accomplished  prior 
to  structuring  a  viable  acquisition  alternati ve.^ 
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Table  4-1 
LANTIRN 

Leader/Follower  (L/F)  Alternatives 


Potential  Savings 


Alternatives 

System 

Production 

Rate 

L/F 

Savings 

L/F 

Qual  Costs 

Potenti al 

Net  Savings 

TY$ 

TY$ 

TYS 

*1 -L/F  Only 

Target  Pod 

16/Month 

309. 8M 

176. 6M 

133. 2M 

-L/F  W/8uy-out 

Target  Pod 

16/Month 

402. 5M 

176. 6M 

225. 9M 

*1 I a-L/F  Only 

Target  Pod 

16/Month 

301 .0M 

182. 1M 

1 1 3 . 9M 

I Ib-L/F  Only 

Target  Pod 

12/Month 

307. 8M 

163. 0M 

139.8M 

I I I a-L/F  Only 

Both  Pods 

16/Month 

449. 4M 

231. 5M 

217. 9M 

IIlb-L/F  Only 

Both  Pods 

12/Month 

501. 0M 

225. 8M 

275. 2M 

*  Difference  is  for  Alternative  I,  competitive  split  began  at  lot  2;  for 
Alternative  Ila,  competitive  split  began  at  lot  3. 


This  analysis  projected  a  downward  shift  of  20  percent  in  the 
learning  curve  and  a  rotation  to  64  percent  from  a  92-percent  curve. 
When  questioned  by  the  government,  the  TASC  analysts  could  not 
reconstruct  the  analysis.  And  a  subsequent  TASC  analysis  showed  a  shift 
of  the  learning  curve  of  9  percent  and  a  rotation  to  88  percent. 9 

As  a  result  of  the  TASC  report,  the  program  office  issued  a  Cost 
Alert  List  (CAL)  projecting  a  savings  of  $139  million  in  base-year  1980 
dollars  and  $320  million  in  then-year  dollars.  On  3  August  1983  ASO/ACC 
reevaluated  the  proposed  savings  but  could  not  validate  them  for  several 
reasons.  First,  they  believed  data  used  to  compute  TASC  cost-estimating 
relationships  were  obtained  from  systems  procured  in  large  quantities 
(up  to  10  times  more)  and  lower  unit  cost  (IS  to  over  100  times  less 
expensive)  and  they  stated  conclusions  from  these  data  could  not  be 
extrapolated  to  LANTIRN.  Second,  the  subsequent  TASC  analysis  reduced 
the  base-year  savings  estimate  to  $5  million.  Third,  TASC  reduced  the 
program  costs  based  on  decreased  systems  engineering/program  management 
and  data  requirements.  ASD/AC  did  not  accept  these  reductions.  Fourth, 
TASC  imputed  a  reduction  of  27  percent  of  recurring  hardware  savings  for 
reduced  general  and  admi ni st rat i ve  (GSA )  expense  and  fee.  However,  the 
G&A  expense  and  fee  in  the  baseline  cost  estimates  were  already  included 
in  the  recurring  cost  savings.  Thpse  adjustments  changed  the  TASC 
estimate  to  a  net  increase  in  base-year  30  dollars  of  $38  million. 


ASD/AC  reestimated  competitive  savings  using  methodology  similar  to 
that  employed  on  the  Aeronautical  Systems  Division  (ASD)  IIR  Maverick 
ICA.  ASQ/ACC  projects  an  additional  $106  million  (FY  30$)  would  be 
required  as  shown  in  table  4-2. 


Table  4-2 

ASO/ACC  LANTIRN  ICA  (720  Units) 
(BY80S,  Millions) 


With 

Competi tion 

W i thout 
Competi tion 

Savings 

Recurring  Hardware 

Leader  (MMC) 

$776 

Fol lower 

250 

Subtotal 

$1,026 

$1 ,045 

$  19 

Other  Costs 

704 

576 

-128 

Engi neeri ng  Change 

Orders 

104 

_ 10  7 

_  _  _3_ 

TOTAL 

$1,834 

$1,728 

$-106  (NET 

Neither  estimate  considered  the  added  costs  to  the  government 
for  selecting  the  second  source,  monitoring  technical  transfer,  or 
administering  the  additional  contracts.  They  also  did  not  address  the 
potential  impact  on  company-funded  RAD,  overhead  impacts,  or  the  benefits 
to  be  gained  through  competition  for  required  spare  parts.  In  addition, 
the  estimates  did  not  discount  cash  flows. 


AMRAAM 

The  advanced  medium  range  air-to-air  missile  (AMRAAM)  was  developed 
by  the  Hughes  Aircraft  Company.  An  estimated  twenty  thousand  missiles 
will  be  built  over  the  nine-year  period  ending  in  1992.  Competition 
between  Hughes  and  Raytheon  was  examined  as  a  potential  acquisition 
strategy  using  the  leader/follower  technique  with  three  years  of 
directed  buys  and  competition  involving  four  additional  lots. 

Analysis  of  the  potential  impact  of  program  costs  under  competitive 
leader/fol lower  procurement  was  undertaken  to  study  the  cost  impact  of 
competition  on  the  air  vehicle.^  According  to  comptroller  personnel, 
the  analysis  was  undertaken  in-house  because  TASC's  analysis  was  not 
acceptable. *  * 
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Armament  Division  (AD)  developed  a  competition  interactive  computer 
model  that  used  parameters  developed  from  Sparrow  and  Sidewinder  data  to 
generate  total  estimated  competitive  cost  for  AMRAAM.  This  was 
accomplished  by  an  analogous  extrapolation  to  future  AMRAAM  lots  of  the 
Sparrow  and  Sidewinder  data  to  Hughes  proposals  for  AMRAAM  lots  i  and 
II.  AD  felt  that  this  was  reasonable  since  both  Sparrow  and  Sidewinder 
programs  were  sole-source  procurements  followed  by  competitive 
procurements.  Proposal  data  for  these  procurements  were  available  for 
seven  lots,  and  data  were  submitted  for  five  different  production 
quantities  or  step  bids  within  each  lot. 

The  analysis  of  the  proposal  data  involved  four  cost  elements: 
material,  direct  manufacturing  labor,  manufacturing  support  labor,  and 
engineering  support  labor.  Noncompetitive  and  competitive  recurring 
cost  trends  were  identified  between  production  lots  and  for  each  bid 
quantity  within  a  production  lot  using  cost-improvement  curve  analysis 
for  each  cost  element.  Nonrecurring  costs  were  estimated  from  AMRAAM 
program  office  sole-source  estimates. 

An  analysis  of  the  cost-improvement  curves  for  the  submitted  step 
bids  for  each  lot  showed  that  the  curves  had  a  range  that  could  reflect 
(a)  contractor  competitive  bidding  strategy,  (b)  cost  improvement,  nr  (c) 
production  rate  effects.  No  attempt  was  made  to  separate  the  individual 
impact  of  these  variables  on  the  data. 

Analysts  calculated  the  competitive  cost-improvement  curve  in  two 
steps.  First,  they  calculated  the  projected  bottom  point  on  the 
cost-improvement  curve  for  each  cost  element  (where  the  cost  improvement 
stops)  by  using  the  ratio  of  the  cost  element's  lowest  observed  cost  or 
hours  competitive  value  to  its  value  at  the  last  noncompetitive 
procu rement: 

lowest  observed  value 

Lowest  value  percentage  =  - 

value  at  last  non¬ 
competitive  procurement 

The  lowest  observed  value  for  Hughes  was  derived  by  multiplying  this 
percentage  by  the  cost  element's  value  proposed  by  Hughes  in  its  last 
noncompetiti ve  proposal.  The  competitive  improvement  curve  for  the  cost 
element  was  then  derived  by  projecting  what  the  improvement  rate  must  be 
to  achieve  the  lowest  expected  value  in  the  penultimate  competitive 
procurement. 

The  actual  improvement  slopes  and  bottom-out  points  for  the  four 
cost  elements  were  calculated  for  Sparrow  and  Sidewinder  from  historical 
data.  Applying  these  parameters  to  the  Hughes  (AMRAAM)  proposal  for 
lots  I  and  II,  analysts  projected  that  the  estimated  recurring  costs  for 
the  sole-source  production  of  20,000  missiles  at  a  rate  of  2,500  per 
year  would  be  $2,284.7  million  in  FY  78  dollars.  They  estimated  that 
the  recurring  cost  using  the  leader/fol lower  strategy,  tooling  each  at  a 
capacity  of  1,800  per  year,  would  be  $2,119.4  million  in  FY  78  dollars. 
Nonrecurring  costs  were  taken  from  SP0  estimates  as  follows: 


FY  78 
mi  1 1 i ons 


fol lower  planning 

$10.4 

Hughes  technical  transfer 

4.0 

reprocurement  data 

12.2 

follower  qualification  program 

8.9 

tooling  and  test  equipment 

49.4 

vendor  nonrecurring  qua  1 i f i cat i on 

_n._i_ 

TOTAL 

$96.0 

Finally 
the  reduced 
year  at  both 

,  analysts  developed  an  estimated  cost  of  $11.8  million 
tooling  required  due  to  the  production  rate  of  1,800 
contractors  compared  to  a  capacity  of  2,500  at  Hughes. 

Table  4 

-3  summarizes  results  of  the  analysis. 

12 

Table  4-3 

Summary  Cost 

FY  78  Millions 

Sole-Source 

Leader/Fol lower 

Additional  nonrecurring  0 

96 

Recu  rri ng 

2,284.7 

2,119.4 

SPO  costs 

546.1 

546.1-11.8=534.3 

TOTAL 

2,830.8 

2,749.7 

for 

per 


Total  undiscounted  estimated  savings  was  81.1  million  dollars  or  2.8 
percent  of  program  costs.  While  the  analysis  presented  the  required 
budget  profile  by  year,  it  did  not  discount  the  differing  profiles  of 
the  sole-source  and  competitive  situations.  Discounting  the  profile 
using  the  10  percent  rate  as  required  by  0MB  circular  A-94  yields  an 
estimated  discounted  savings  of  $6.1  million  as  shown  in  table  4.4. 
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This  value  is  obtained  by  multiplying  the  undiscounted  sole-source  and 
leader/follower  figures  by  th  discounting  factor,  summing  the 
respective  products,  and  subtracting  for  the  difference.^ 

In  addition  to  not  discounting  the  analysis,  the  AD  analysts  did 
not  consider  the  impact  of  changes  on  other  stages  of  the  acquisition 
process  or  on  other  programs.  Moreover,  they  felt  there  would  be  a 
reduction  in  SPO  costs  and  did  not  consider  contract  administration  or 
additional  configuration  management  costs. 


MLRS 


The  multiple  launch  rocket  system  (MLRS)  is  a  free-flight  rocket 
system  consisting  of  a  self-propelled  loader  launcher  (SPLL)  and  two 

expendable  launch  p< ds/containers  (LP/C),  each  of  which  stores  six 

rockets.  The  rockets  contain  a  solid  propellant  motor,  fuze,  and 

wa  rhead . 

Five  companies  were  awarded  contracts  for  concept  definition  in 
1976,  with  Boeing  and  Vought  being  awarded  competitive  development 
contracts  in  1977.  In  1980  Vought  was  chosen  for  final  development  and 
awarded  contracts  for  production  facilities  and  four  options  for  low- 
rate  production.  These  options  totaled  28,476  rockets  and  4,746  LP/Cs. 

MLRS  DSARC  III  guidance  directed  that  the  Army  consider  acquisition 
strategies  to  obtain  production  price  competition  on  the  rocket  and 

LP/C  and  that  it  consider  the  impact  on  price  of  different  total  program 
quantities  and  production  rates.  The  Army  completed  its  second-source 
acquisition  analysis  in  December  1980.^  The  study  was  designed  to 
provide  estimates  of  the  optimum  procurement  strategy  for  either  190,000 
or  360,000  rockets  and  a  break-even  point  for  each  strategy. 

The  study  evaluated  four  competitive  options.  The  technical  data 
package  (TOP)  option  involved  obtaining  a  validated  data  package  from 
the  developer  and  soliciting  from  other  contractors.  The  second  option 
involved  the  leader/follower  concept  in  which  technical  data  is  transferrei 
to  the  second  source  during  development.  A  third  option  was  labeled  the 
"freedom  of  design  approach"  (FOD)  in  which  Boeing  would  build  a 
LP/C-rocket  of  its  own  design  but  capable  of  i nteroperabi 1 i ty  with 
Vought 's  SPLL.  The  fourth  option  involved  a  freedom  of  design  approach 
in  which  the  second  source  was  provided  a  copy  of  Vought's  unvalidated 
TOP  and  allowed  considerable  design  flexibility  in  making  changes. 
Production  rate  options  under  each  strategy  involved  2,000,  4,000,  and 
6,000  units  produced  per  month. 

Each  combination  of  procurement  strategy,  production  rate,  and 
production  quantity  was  evaluated  under  three  criteria:  program  and 

contractual  issues,  technical  and  operational  issues,  and  economic 
issues.  Program  and  contractual  issues  involved  the  quantitative 
assessment  of  schedule  impacts,  configuration  management,  warranties, 
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prime  contractor  cooperation,  and  the  impact  on  provisions  of  a 
memorandum  of  understanding  between  participating  countries  in  the 
procurement.  Technical  and  operational  issues  included  the  potential 
availability  of  second-source  contractors,  potential  for  technological 
innovation,  impact  on  the  fire  control  system  integral  to  the  SI’LL, 
logistic  considerations,  assessment  of  technical  risk,  testing 
requi rements,  and  TOP  validation  requirements.  Economic  issues  included 
analysis  of  increased  nonrecurring  and  decreased  recurring  costs 
calculated  as  a  percentage  reduction  of  projected  sole-source  cost. 

Sole-source  cost  estimates  were  based  on  a  first-unit  cost  of 
$10,726  (FY  80)  for  a  production  rate  of  6,000  per  month.  Reduction  in 
quantity  from  360,000  to  190,000  was  estimated  to  cause  a  4  percent 
increase  in  unit  cost  due  to  capital  depreciation  over  the  smaller 
quantity.  Reduction  in  production  rate  from  6,000  rockets  per  month 
increased  first-unit  cost  as  follows: 


Rate  per  month 

First-unit  cost  increase 

4,000 

4% 

3,000 

10% 

2,000 

25% 

Unit  costs  were  projected  to  increase  by  6  percent  after  FY  33  to 
reflect  noncompetition  in  follow-on  procurements. 

Nonrecurring  costs  differed  between  acquisition  strategies. 
Included  in  the  f reedom-of-desi gn  option  were  additional  research  and 
development,  TOP  validation,  government  R&D,  and  government-furnished 
equipment  and  testing.  Additionally,  all  competitive  options  included 
projected  costs  for  contractor  system  and  production  engineering  and 
program  management,  prime  contractor  support  to  the  second  source, 
production  qualification  testing,  and  initial  production  facilities  cost. 
Table  4-5  shows  the  estimated  lower  and  upper  limits  for  each  cost 
element  and  acquisition  strategy.^ 

Recurring  costs  were  estimated  for  the  following  elements  in  each 
option:  hardware  manufacturing,  recurring  engineering,  quality  assur¬ 

ance,  engineering  changes,  data,  contractor  and  government  system 
project  management,  and  first  destination  change. 

Hardware  manufacturing  cost  accounted  for  65.2  to  79.7  percent  of 
total  cost  of  the  options  and  was  given  the  most  attention.  Three 
variables  were  used  to  estimate  recurring  manufacturing  cost: 
theoretical  first-unit  cost,  learning  curve  slope,  and  learning  curve 
step  functions. 


Table  4-5 

MLRS  Nonrecurring  Cost  Summary 
(FY  80  $M) 


Option 

2, 000 /Mo 

1,000 /Mo 

4',000/Mo 

6,000/Mo 

Sole-Source 

20 

25 

30 

38 

TOP  Traditional 

39-57 

46-66 

52-75 

62-89 

TDP  LDR/F0L 

45-68 

52-77 

58-86 

68-100 

F0D  Designated 

94-124 

101-130 

108-136 

118-145 

F0D  Competitive 

106-137 

114-146 

121-155 

131-169 

t  theoretical  first-unit 

cost  of  $6,849 

(FY  80$) 

for  the  MLRS 

rocket  (less  submunitions  and  fuze)  and  $18,229  for  the  LP/C  were  used 
in  the  DSARC-approved  baseline  cost  estimate  and  were  accepted  by  the 
study  team. 16  This  estimate  was  predicated  upon  a  maximum  production 
rate  of  6,000  rockets  per  month  and  a  total  production  quantity  of  at 
least  326,475.  As  all  options  did  not  meet  this  baseline,  first-unit 
cost  was  adjusted  to  reflect  changes  in  production  rate  and  total 
quantity.  Rate  adjustments  were  as  follows: 

Rocket  Rate  Relative  Unit  Cost 


2,000 

1.25 

3,000 

1.10 

4,000 

1.04 

6,000 

1.00 

Learning  curve  slope  was  based  on  a  91  percent  curve  for  both  the 
prime  and  second  source  as  developed  for  the  DSARC  review.  Factors  that 
supported  the  shallow  91  percent  curve  were  the  high  degree  of 
automation  and  the  fact  that  material  comprises  over  70  percent  of  total 
recurring  manufacturing  cost. 

Competitive  pressures  on  recurring  cost  were  treated  as  learning 
curve  step  functions.  Four  step  functions  accounted  for  the  initial  low 
production  rate  change  expected  for  split  competition,  change  expected 
for  a  buy-out  competition,  and  a  percent  change  expected  when  reverting 
from  previously  competitive  to  noncompetitive  procurement.  Each  option 
had  a  unique  combination  of  these  step  functions. 

The  initial  low  rate  upward  adjustment  was  made  to  projected  costs 
f^om  the  learning  curve.  The  following  factors  applied: 
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Percent  Increase 

Production  1st  year  2nd  year  3d  year 

6,000/ino  30.3  20.2  23.5 

3,000/mo  IB. 7  9.5  12.4 

These  factors  reflected  increased  unit  costs  because  the  automated  1 i n*-s 
were  not  operating  at  design  capacity. 

Cost  improvement  associated  with  split  competition  was  estimated  at 
7.5  percent  for  TOP  options  and  10  percent  for  FOD  options.  These  rates 
were  judgment  estimates  based  on  the  APRO  79  study  of  22  systems  and  the 
consulting  report  by  Putnam,  Hayes  and  Bartlett  discussed  in  chapter  3. 

The  cost  improvement  associated  with  a  buy-out  was  chosen  as  15 
percent  for  TOP  options  and  20  percent  for  FOD  options.  This  judgement 
was  based  on  three  sources  discussed  earlier: 

•  APRO  78 

--  16  systems  studied:  Net  Recurring  Savings  -13%  to  +51% 

Mean  Savings  of  13.7% 

•  Institute  of  Defense  Analysis  79 

--  31  systems  studied:  Gross  Savings  -23%  to  +64% 

Mean  Savings  of  36.1% 

•  APRO  Letter,  3  October  80 

--  Specific  recommendations  for  this  study: 

Gross  Savings:  10%  TDP  options 

15%  Freedom-of-desi gn  options 

The  final  step-adjustment  was  made  for  the  transition  from  a  com¬ 
petitive  situation  back  to  a  noncompetitive  situation  after  the  buy-out. 
A  figure  of  +5  percent  was  estimated  to  reflect  this  decreased 
competitive  pressure. 

The  basis  for  recurring  cost  calculation  is  the  applicable  first- 
unit  cost  and  91  percent  learning  curve  multiplied  by  the  option's 
unique  step  functions.  Figures  4-1  and  4-2  depict  these  step  functions 
over  time  as  they  affect  each  baseline  unit  cost  estimate.” 

The  LRP  baseline  on  the  figures  represents  the  low-rate  production 
cost  baseline  for  sole-source  production,  and  the  balance  of  the  figures 
graphically  illustrate  the  impact  on  the  baseline  price  of  the  four  step 
functions. 


UNIT  COST  RELATIVE  TO  BCE  (%)  UNI  I  COST  RLLATIVE  TO  BCE  (%) 


Figure  4-1.  Assumed  Effects  of  Competition  on  Unit  Cost  (TDP  Copy) 


Figure  4-2.  Assumed  Effects  of  Competition  on  Unit  Cost 
(Freedom  of  Design) 

The  study  derived  other  cost  elements  by  applying  historical  factors, 
developed  by  MICOM  from  previous  program  histories,  against  the  baseline 
cost  estimate  for  hardware  manuf acturing  cost. 

The  analysis  is  calculated  in  FY  80,  then-year,  and  constant  dollars 
discounted  at  10  percent  in  accordance  with  0MB  Circular  A-94 
procedures.  After  evaluating  the  options,  the  study  concluded  there  was 
no  single  acquisition  strategy  that  was  dominant  on  every  issue.  Using 
the  calculation  in  FY  80  dollars,  it  concluded  that  15  of  the  40 
competitive  options  were  more  favorable  than  sole  source.1®  The  most 
favorable  option  for  a  total  quantity  of  190,000  rockets  was  the 
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sole-source  option  at  6,000  per  month  and  the  leuder/fol lower  rate  <>f 
6,000  rockets  per  month  for  a  total  quantity  of  360,000  was  the  mist 
economical.  Based  on  program  and  contractual  issues  together  with  the 
technical  and  operational  issues,  the  study  recommended  a  DP 
traditional  strategy. 

Analysis  in  FY  80  dollars  discounted  at  10  percent  showed  that 
only  two  of  the  40  combinations  of  options  and  production  rates  had 
potential  savings  at  the  total  cost  level.  Both  were  at  a  total 
production  of  360,000  using  the  TOP  leader/fol lower  method  at  a  rate  of 
either  3,000  or  6,000  per  month. ^ 

The  team  developed  a  probability  distribution  of  competitive 
savings  to  account  for  the  economic  uncertainty  of  the  cost  estimates. 
It  accomplished  this  by  subjectively  assigning  probabilities  that  the 
cost  of  a  specific  option  would  be  less  than  or  equal  to  the  sole-source 
cost.  The  distributions  also  favored  the  TP  leader/follower  approach  as 
having  the  greatest  probability  of  savings. 

The  study  recommended  that  a  total  quantity  of  360,000  rockets  be 
procured  at  the  rate  of  6,000  per  month  using  a  TDP  traditional  approach 
based  on  factors  other  than  cost. 


Bradley 

The  Bradley  fighting  vehicle  system  (FVS)  is  a  mechanized  infantry 
combat  vehicle  developed  by  the  FMC  Corporation  starting  in  1977.  As  of 
November  1982  FMC  had  received  three  production  awards  totaling  1,700 
vehicles  out  of  a  proposed  total  baseline  quantity  of  6,882  vehicles  to 
be  procured  through  FY  89. 

A  1982  study  reassessed  the  costs  and  benefit  of  competitive 
strategies  designed  to  reduce  rapidly  escalating  program  costs. ^  The 
reassessment  examined  four  prior  studies  of  the  cost  effectiveness  of 
competing  the  FVS. 

It  compared  the  sole-source  costs  of  two  competitive  strategies. 
One  strategy  involved  split  awards  for  the  duration  of  the  program.  The 
other  was  a  buy-out  where  a  "wi nner-take-al  1 11  competition  is  held  after 
two  years  of  split  awards. 

Competition  would  be  based  on  a  technical  data  package  and  would 
encompass  the  manufacturing  and  assembly  of  the  vehicle,  which  repre¬ 
sents  45  percent  of  unit  cost.  The  engine,  turret  drive,  transmission, 
fire  control,  and  other  weapons  would  be  government-furnished  equipment. 
Analysis  of  the  competitive  options  included  examining  nonrecurring  and 
recurring  costs  as  well  as  noncost  issues. 

Nonrecurring  costs  were  divided  into  start-up  costs  and  an  initial 
production  penalty  cost.  Start-up  costs  included  additional  vendor 


tooling,  machinery,  facilities,  traininq,  and  engineering  costs  and  were 
taken  from  firm-fixed  price  contractor  submission  or  second- source 
budgetary  estimates.  Start-up  costs  for  a  second- source  production 
capability  of  50  per  month  were  estimated  to  cost  $32.20  million  in 
December  80  dollars.  An  increase  to  90  percent  per  month  would 
require  an  additional  $10.85  million.  The  initial  production  penalty 
cost  estimate  ranged  from  43.7  to  45.5  million  dollars,  depending  on  the 
strategy.  It  represents  an  amount  calculated  by  subtracting  the 
expected  FMC  price  to  produce  the  23^  vehicles  in  FY  83  and  FY  84  and 
start-up  capacity  costs  from  estimated  second- source  prices  for  the  974 
vehicle  education  buy  in  FY  83  and  FY  84. 

Recurring  costs  included  contract  hardware  prices,  government 
administrative  costs,  and  other  program  costs  and  spillover  effects. 
Contract  prices  included  all  contractor  costs  since  a  single  contract 
was  being  contemplated.  They  were  calculated  using  a  quantitative  model 
based  on  unit  learning  curve  theory  of  the  formulation: 


_  (b-o,) 

c, «  o.o,  2.  *q 

q * q.  * •  l 


Where 

Cj  =  total  recurring  cost  through  i  th  i  tan 
a  =  f i rst- uni t  cost 
b  =  rate  of  cost  improvement 
g-j  =  discount  factor  for  year  i 
q-j  =  production  experience  factor  at  end  of  ^ear  i 
Pi  =  percent  reduction  in  unit  cost  at  year  i  due  to  competition 
o  i  =  relative  chanqe  in  price  performance  improvement  rate  at  year 
i  due  to  competition 

This  formulation  can  accommodate  a  potential  shift  in  the  learning 
curve  or  an  improvement  in  the  learning  rate  (learning  curve  rotation). 
It  can  also  be  adjusted  for  production  rate  and  accounts  for  the  time 
value  of  money  by  discounting  future  dollars.  Any  one  or  all  the 
adjustments  can  be  made  for  any  given  production  year. 

Estimates  of  future  >ear  impacts  are  shrouded  with  uncertainty. 
Therefore,  the  analysis  accounted  for  uncertainty  throuqh  the  use  of  an 
eve  nt-  ori  ented  Monte  Ca^lo  simulation.  Individual  values  for  each 


factor  were  given  minimum,  maximum,  and  most  likely  magnitudes  based  on 
the  analysts'  judgement.  By  treating  each  factor  in  this  manner,  the 
analysts  developed  a  cumulative  probability  distribution  for  ^ach 
strategy . 

The  sole-source  learning  curve  slope  was  estimated  to  follow  a  93 
percent  slope  under  all  competitive  conditions  for  units  1100-6382, 
consistent  with  the  baseline  cost  estimate.  However,  evidence  from 
other  vehicle  programs  revealed  a  95  percent  slope,  and  analyses  were 
run  using  both  slopes.  The  analysts  did  not  expect  a  rotation  of  the 
learning  curve  as  a  result  of  competition,  but  they  did  expect 
competitive  pressures  to  shift  the  learning  curve  and  the  production 
rate  to  impact  each  strategy  as  follows: 


Competitive  Production  Rate  Overall 


Strategy 

Reduction 

Penal  ty_ 

Shi  ft 

Range 

Multiyear  Buy-out 

15 

0 

15 

+  10 

Multiyear  Spl it-buy 

10 

-3 

7 

+  7 

Single  Year  Spl it-buy 

8 

-3 

5 

+  7 

The  analysts  estimated  government  program  office  admi ni st rat i ve 
costs  to  increase  $2  million  per  year  and  additional  testing  to  cost  at 
.5  million  per  year  for  three  years.  Effects  on  logistics,  maintenance, 
and  engineering  sectors  were  considered  minimal  and  no  additional 
contract  administration  was  deemed  necessary.  Spillover  effects  on 
other  government  programs  were  considered  and  dismissed  as  balanced  out 
between  contractors.  Other  costs  discussed  and  quantitatively  dismissed 
included  contractor  systems  technical  support,  contractor  data 
requirements,  government  furnished  equipment,  product  improvements,  and 
the  impact  on  spares  procurement. 

The  data  shown  in  table  4-6  are  the  result  of  inputting  the 
recurring  cost  values  for  the  parameters  into  the  computer  model  of 
price  behavior  for  a  production  rate  of  1,080  vehicles  per  year  and  then 
combining  the  results  with  nonrecurring  costs  for  all  strategies.^ 

The  analysis  concluded  that  the  competitive  strategies  will  never 
show  significant  savings  in  discounted  dollars.  For  the  dual-source 
multiyear  strategy  on  a  93  percent  curve,  expected  i ncrease  in  outlays 
was  22.2  million  (14.4  million  undiscounted  FY  81  dollars!-  For  the 
multiyear  buy-out  strategy  the  analysts  projected  a  net  savings  of  S19 
million.  While  the  analyses  were  run  assuming  a  production  rate  of 
1,080  a  year,  the  approved  baseline  quantities  are  not  currently  that 
large  and  the  analyses  showed  that  competition  was  not  economically 
feasible  at  the  lower  quantity  using  any  strategy. 

A  multiyear  strategy  was  also  calculated  using  cancellation 
ceilings.  Model  input  factors  included:^ 
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Expanded  Multi-Year  Input  Values 


Contract  Situation 

Reduct  ion 

P  roduct i on 
Rate 

Shi  ft 

Range 

Sole-Source 

n 

0% 

n 

+  3% 

Competitive  Buy-Out 

19% 

0% 

19% 

+  10% 

Competitive  Split-Buy 

14% 

3% 

11% 

+  7% 

Using  POM  quantities,  sole-source  savings  of  $75.9  million  dollars 
($104.0  million  discounted)  were  projected. 

The  authors  considered  noneconomic  issues  along  with  potential 
economic  savings.  One  identified  benefit  was  better  cooperation; 
another  was  improved  contract  terms  from  the  sole-source  contractor. 
They  identified  as  negative  factors  the  probable  schedule  slippage  in 
the  second  source  and  the  poo.r  condition  of  the  TDP  which  might  cause 
claims  after  award.  Additionally,  the  sole  source  is  not  providing  some 
drawings  and  process  data  because  they  allege  that  this  is  proprietary 
data.  Also  no  costs  are  included  for  testing  of  the  product  by  the 
government,  and  configuration  management  will  be  a  problem  because  the 
design  will  still  he  undergoing  changes  at  the  time  of  contract  award. 
And  there  is  a  potential  one-year  slip  in  delivery  reflecting  the 
cumulative  effect  of  these  problems. 

The  authors'  recommended  strategy  was  a  sole-source,  multi-year 
award.  They  noted  that  this  also  has  the  least  program  risk  but  leaves 
the  developing  contractor  in  a  favorable  position.  Accordingly,  they 
recommended  a  "red  team"  contract  to  a  consultant  who  will  monitor  the 
incumbent's  progress  and  recommend  cost-reduction  opportunities. 


HARM 


The  high-speed  anti  radiation  missile  (HARM)  was  developed  by  Texas 
Instruments,  with  the  warhead  consisting  of  2,500  preformed  fragments. 
It  can  be  fired  in  three  different  modes:  self-protection,  target  of 

opportunity,  and  standoff  modes.  Total  procurement  was  expected  to 
involve  21,388  missiles  between  FY  81  and  FY  92. 

The  controversy  surrounding  the  decision  not  to  second-source  '■he 
HARM  highlights  the  inexact  nature  of  estimating  the  impact  of 

competition.  Within  the  Navy,  two  separate  estimates  were 

accomplished,  one  by  the  Naval  Material  Command  (NAVMAT)  and  one  by 
Naval  Air  Systems  Command  (NAVAIR).  Both  were  developed  in  FY  82  dollars 
and  done  because  agreement  could  not  be  reached  on  methodology  and 

assumptions. 23  Additionally,  OSD  Program  Ana  1 ys i s  and  Evaluation  (PA, IE! 
acting  in  its  Cost  Analysis  Improvement  Group  (CAIG)  role,  completed  a 
review  of  the  Navy  analysis. 


104 


Estimates  within  the  Navy  were  developed  individually  for  three 
elements:  (1)  guidance,  control,  wings,  and  fins  (GC&A);  (2)  integration 
and  assembly  (T3A);  and  (3)  program  management  systems  engineering 
(PMSE).  NAVMAT 's  estimate  was  accomplished  at  the  detail  level, 
estimating  labor,  material,  and  profit  for  each  of  these  elements  based 
on  the  HARM  Cost  Study  Group  estimates  for  sole-source  procurement. 
Competitive  impacts  were  calculated  based  on  Sparrow  AIM-7F  experience. 
The  NAVAIR  developed  estimate  for  sole-source  cost  was  also  a  detailed 


estimate 

developed  by 

the  HARM  Cost  Team. 

Their  competition  estimate 

was  done 

,  however,  at 

the  bottom  line  price 

.  Estimates 

at  the  bottom 

line  for 

unit  cost  for 

both  are  shown  in  table  4-7. 

Table  4-7 

NAVMAT  and  NAVAIR 

Estimates  of  HARM  Costs 

FY-82  ( $000 ' s ) 

NAVMAT  Unit  Costs 

Competition 

Sole-Sou  rce 

:  P  r  i  me 

Second 

Total 

Quantity 

21,392 

13,860 

7,532 

21,392 

GC4A 

132.0 

134.9 

161.4 

144.2 

ISA 

12.1 

12.4 

15.0 

13.3 

PMSE 

10.9 

13.5 

8.2 

11.7 

Total 

155.0 

160.8 

184.6 

169.2 

NAVAIR  Unit  Costs 

Quantity 

21,392 

11,082 

10,309 

21,391 

GC&A 

130.6 

ISA 

12.2 

PMSE 

10.0 

Total 

152.8 

162.2 

134.3 

148.8 

While  NAVAIR  showed  a 

net  savings,  NAVMAT 

showed  an  increase  of  the 

competitive  price  over  the  sole-source  price.  The  sole-source  estimates 
are  very  close  to  each  other.  The  principle  difference  is  reflected 
in  the  cost  of  the  second  source.  The  official  NAVY  position  at  DSARC 
was  that  there  would  be  a  net  savings  resulting  from  second-sourcing. 

OSD  CAIG  reviews  of  the  HARM  program,  using  different 
methodologies,  arrived  at  different  conclusions  about  the  benefits  of 
second  sourcing.  Disagreements  surfaced  about  learning  curve  slopes, 
special  test  equipment,  facilities,  and  other  contractor  investments. 
Additionally,  reviewers  could  not  identify  which  cost  elements  would  be 
reduced  and  no  contractual  guarantees  could  be  obtained  to  fix  the  price 
of  the  out-year  requirements.  Accordingly,  the  secretary  of  defense 
directed  that  the  program  be  procured  sole  source.  ^ 
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Discussion 

Analysis  of  the  program  specific  estimates  of  the  impact,  of 
competition  reveals  the  use  of  disparate  approaches  to  accomplish  toe 
estimates.  The  studies  differ  in  several  aspects:  a)  cost  elements 
considered,  b)  depth  of  analysis  of  each  element,  c)  whether  point 
estimates  or  ranges  are  used  to  account  for  uncertainty,  d)  whether  cash 
flows  are  discounted,  and  e)  the  extent  the  studies  relied  on  less  than 
robust  historical  studies.  Because  of  these  differences,  the  confidence 
that  can  be  placed  in  their  conclusions  also  varies. 


Factors  Considered 

Chapter  2  identified  factors  as  being  quantifiable  and  nonquantifi- 
able.  Quantifiable  factors  include  nonrecurring  and  recurring  costs, 
profit,  and  other  quantifiable  costs.  Nonquanti f i able  factors  include 
cooperation,  claims,  product  quality,  time  delays,  and  surge  capability. 
Table  4-8  is  a  matrix  that  identifies  each  program  and  the  factors 
considered  in  each  analysis.  Most  of  the  potential  impacts  were  treated 
in  the  studies  with  two  notable  exceptions.  Excluded  were  those 
government  costs  incurred  to  select  and  monitor  the  second  source  and 
those  costs  and  benefits  that  would  impact  on  other  stages  of  the 
acquisition  process. 

The  incremental  costs  directly  incurred  by  the  government  to  select 
and  monitor  the  second  source  were  the  most  frequently  omitted  factors. 
Specifically,  the  second-source  selection  costs  were  not  considered  in 
any  analysis  except  the  MLRS  study.  Moreover,  the  government  costs  to 
monitor  and  assist  technology  transfer  during  second-source  development 
and  the  additional  contract  admi ni st rat i on  costs  during  contract 
performance  were  not  considered  in  any  case  except  those  of  the  Bradley 
and  MLRS  analyses.  Additionally,  these  two  studies  were  the  only  ones 
that  addressed  the  added  cost  of  configuration  management  necessary  to 
ensure  that  the  second  source  receives  any  change  made  to  production 
drawings  or  processes  by  the  developing  contractor. 

The  other  quantifiable  factors  most  frequently  omitted  were  those 
that  would  be  incurred  in  stages  of  the  acquisition  process  other  than 
the  production  stage  under  immediate  consideration.  For  example,  the 
MLRS  analysis  was  the  only  analysis  considering  the  impact  on 
company-funded  R&D  during  the  concept  definition  and  validation  stages. 
And  the  Bradley  analysis  was  the  only  one  considering  logistics  impacts 
on  the  system  under  consideration  and  the  overhead  absorption  impact  on 
other  contracts  within  the  profit  center  affected  by  the  reduced 
allocation  base.  Finally,  none  of  the  analyses  considered  benefits 
accruing  to  the  government  during  weapon  system  deployment. 
Specifications  and  drawings  owned  by  the  government  would  allow  the 
government  to  obtain  the  benefits  of  competition  during  spare  parts 
procurements  in  the  operations  and  support  phase  of  system  deployment. 
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Table  4-T 

F]prpor|1_c  f~  ryic;  i  hy  0  p-,  ft>~  <\  1  y~  03 


MLRS 

BFV 

AMRAAM 

HR 

MAVERICK 

V* 

02 

M 

H 

<C 

Quantifiable 

Nonrecurring 

Second-Source  Selection 

Y 

N 

N 

N 

N 

Second-Source  Development 

Technical  Data  Pkg 

Purchase 

Y 

Y 

Y 

Y 

Y 

Validation 

Y 

Y 

Y 

Y 

Y 

Special  Tooling 

Y 

Y 

Y 

Y 

Y 

Special  Test  Equip 

Y 

Y 

Y 

Y 

Tech  Transfer 

Leader -Follower 

Y 

N/A 

Y 

Y 

Y 

Education  Buys 

Y 

Y 

Y 

Y 

Y 

Govt  Monitoring 

Y 

Y 

N 

N 

N 

First  Article 

Y 

Y 

Y 

Y 

Y 

Contingent  Liabilities 

Y 

N/A 

N 

N 

N 

Recurring 

Cost/Quantity  Relationship 

Production  Rate 

Y 

Y 

Y 

N 

Y 

Learning 

Y 

Y 

Y 

Y 

Y 

Contract  Administration 

Y 

Y 

N 

N 

N 

Configuration  Mgt 

Y 

Y 

N 

N 

Y 

Other  Costs 

Company- Funded  R&D 

Y 

Y 

N 

N 

N 

Spare  Parts 

N 

Y 

N 

N 

N 

Overhead  Impact 

N 

Y 

N 

N 

N 

Logistics 

N 

Y 

N 

N 

Y 

Prof  it 

Y 

Y 

N 

N 

N 

Nonquant if iable 

Productivity 

Product  Quality 

Y 

Y 

Y 

N 

Y 

Cooperation 

Technical  Transfer 

Y 

Y 

Y 

N 

N 

Ongoing 

Y 

Y 

N 

N 

N 

Time  Delays 

Y 

Y 

N 

N 

Y 

Claims 

Y 

Y 

N 

N 

N 

Statistical 

Inflation  Adjust 

Y 

Y 

Y 

Y 

Y 

Discounting 

Y 

Y 

N 

N 

N 

N=NO  Y=YES 
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The  potential  nonquanti tati ve  costs  and  benefits  were  not 
considered  except  in  the  Bradley  and  MLRS  analyses.  Excluding  these 
impacts  from  analyses  removes  them  from  explicit  recognition  by  the 
decision  maker  in  judging  whether  the  total  quantitative  and  qualitative 
benefits  outweigh  potential  costs  of  second-sourcing. 


Depth  of  Analysis 


Each  estimate  varied  in  the  depth  of  the  analysis  of  identified 
factors:  either  the  estimating  was  done  at  the  total  contract  level  for 
each  factor  or  the  factors  were  disaggregated  into  individual  cost 
elements.  Moreover,  some  of  the  studies  identified  more  detailed  cost 
elements  than  others,  some  projected  cost  impacts  for  the  various 
strategies  of  technology  transfer,  while  others  made  estimates 
independent  of  the  method  of  technology  transfer.  The  more  specific  the 
cost  elements  are  the  more  confidence  that  can  be  placed  in  their 
estimation  because  the  impact  of  competition  can  be  more  readily 
identified.  For  example,  if  price  is  disaggregated  into  material, 
labor,  and  profit,  the  impact  of  competition  on  each  of  these  cost 
elements  is  more  easily  projected  than  on  a  total  contract  price  basis. 
It  is  also  more  easily  projected  when  the  alternative  acquisition 
strategies  are  examined  separately.  The  Joint  Tactical  Information 
Distribution  System  ( J  T I  OS )  estimate  was  done  at  the  contract  level 
using  a  10  percent  rule-of-thumb  reduction  from  the  estimated  sole- 
source  cost  and  did  not  consider  differences  in  the  method  of  technology 
transfer.  The  AMRAAM  analysts'  estimate  for  recurring  costs  was 
disaggregated  into  four  cost  elements--materi al ,  direct  manufacturing 
labor,  manufacturing  support,  and  engineering  support  labor--and  they 
considered  both  the  sole  source  and  the  leader/follower  acquisition 

strategy  in  their  estimates.  The  HR  Maverick  program  was  di saggregated 
into  the  guidance  and  control  section,  center  aft  section,  system 
engineering  and  program  management,  sustaining  engineering  and  factory 
support,  engineering  change  orders,  support  equipment,  and  spare  parts. 
The  analysts  considered  three  scenarios  in  their  analysis.  However,  for 
cost  reductions  attributable  to  competition,  they  took  an  overall  9 
percent  cost  reduction  and  5  percent  fee  reduction.  The  LANTIRN 
estimate  was  taken  at  the  contract  level,  and  the  Bradley  and  MLRS 
estimates  both  estimated  the  impact  of  competition  on  alternative 

strategies.  The  MLRS  analysis  examined  four  separate  acquisition 
options  and  estimated  the  impact  for  each  combination  of  option  and 

production  rate.  The  Bradley  analysis  also  identified  two  options  and 

analyzed  each  independently.  Whereas  the  Bradley  analysis  was 
accomplished  at  the  contract  hardware  price  level,  the  MLRS  analysis  was 
done  at  the  detail  cost  element  level. 


Poi nt  Estimates  vs  Ranges 

Only  the  Bradley  and  MLRS  analyses  attempted  to  deal  with  the 
uncertainty  of  estimates.  Both  developed  probability  distributions 
to  account  for  the  economic  uncertainty  of  cost  estimates.  The  MLRS 
used  a  subjectively  assigned  probability  and  the  Bradley  used  a  Monte 
Carlo  technique  to  set  forth  a  cumulative  probability  distribution. 


Discounting 

The  Army  estimates  were  the  only  analyses  discounted  in  accordance 
with  the  requirements  of  0MB  Circular  A-94.  Consequently,  the  other 
analyses  tended  to  understate  the  true  costs  to  the  government  because  they 
ignored  the  time  value  of  money  by  considering  a  $1  invested  up-front  in 
nonrecurring  costs  the  same  as  a  $1  cost  avoided  several  years  into  the 
procurement. 


Reliance  on  Historical  Data 

Most  of  the  program  studies  relied  on  unweighted  average  savings 
derived  from  historical  studies.  The  JTIDS  10  percent  rule-of-thumb 
decrement  was  based  on  expert  judgement  and  nonattributive  references  to 
prior  studies  that  averaged  greater  than  the  10  percent  figure.  LANTIRN 
and  Maverick  both  relied  exclusively  on  The  Analytic  Sciences 
Corporation  (TASC)  studies  of  historical  data.  AMRAAM  relied  on 
proposal  data  from  the  Sparrow  and  Sidewinder  programs  because  they  did 
not  accept  the  TASC  analysis  done  for  them.  The  MLRS  projected  the  step 
function  adjustment  associated  with  buy-out  based  on  an  average  of 
average  savings  taken  from  the  APRO  7B  and  IDA  79  studies,  and  an  APRO 
letter  of  30  October  1980.  The  Bradley  analysis  projected  shifts  of 
learning  curves  based  on  a  review  of  historical  studies. 


Con cl u  s ions 

It  is  not  possible  to  directly  assess  the  confidence  placed  in  the 
predicted  cost  impact  of  second-sourcing  because  what  never  happened  is 
immeasurable;  it  is  impossible  to  know  what  the  price  would  have  been  in 
the  absence  of  competition.  And  the  differences  of  opinion  surrounding 
the  HARM  analysis  serve  to  highlight  the  differences  of  opinion  that  can 
develop  among  competent  analysts.  However,  the  robustness  of  the 
various  studies  can  be  inferred  from  the  review  of  the  five  aspects  of 
the  studies  just  covered:  factors  considered,  depth  of  analysis,  use  of 
point  estimates  or  ranges,  discounting  of  cash  flows,  and  the  reliance  on 
historical  studies'  unweighted  average  data. 

Each  study  satisfied  each  criterion  to  a  different  degree,  as 
discussed  earlier.  The  Bradley  and  MLRS  analyses  tended  to  consider 
more  of  the  substantive  cost  factors  than  did  the  other  studies.  The 
AMRAAM,  Maverick,  and  MLRS  each  examined  multiple  strategies  separately 
at  the  detailed  cost  level.  The  Bradley  and  MLRS  were  the  only  two 
analyses  dealing  with  uncertainty  and  also  discounting  program  cash 
flows  over  time.  And  finally  the  AMRAAM  study,  although  relying  on 
proposal  rather  than  actual  data,  was  the  only  study  not  relying 
extensively  on  the  unweighted  historical  averages  data. 

No  study  was  able  to  adequately  address  all  criteria.  While  the 
MLRS  and  Bradley  studies  were  the  more  comprehensive  in  projecting  the 
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true  impact  to  the  government  by  considering  most  of  the  cost  factors 
and  discounting  the  cash  flows,  all  studies  relied  on  the  historical 
data  to  more  of  an  extent  than  the  rigor  of  the  historical  studies 
allow,  as  argued  in  chapter  3. 

Chapter  5  will  now  propose  a  framework  that  can  be  used  in 
competition  analysis  that  will  overcome  the  methodological  and  data 
shortcomings  that  presently  do  not  allow  satisfaction  of  the  criteria. 
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CHAPTER  5 


CONCLUSIONS  AND  RECOMMENDATIONS 

The  decision  to  second-source  a  weapon  system  in  its  production 
phase  should  he  made  as  early  in  the  acquisition  process  as  possible  to 
allow  maximum  flexibility  for  choice  of  the  method  of  technology 
transfer  to  the  second  source.  And  the  decision  to  second-source  must 
be  made  in  consonance  with  the  primary  acquisition  program  objectives: 
system  technical  performance,  production  surge  capability,  required 
delivery  schedule,  life-cycle  system  cost,  and  system  supportabi 1 i ty . 
Each  possible  strategy  (such  as  leader/follower  or  f reedom-of-desi gn 
approach)  affects  the  objectives  in  different  ways. 


Several  studies  have  looked  at  acquisition  strategies.  Duane 
Knittle  and  Robert  Williams  coauthored  a  paper  examining  the  systems 
acquisition  planning  phase  used  to  plan  the  strategy  for  achieving 

program  objectives  for  the  entire  life  of  an  acquisition. *  More 
recently  Charles  Smith  and  Charles  Lowe,  Jr.,  coauthored  a  paper  on 
strategy  selection  for  the  production  phase  of  the  acquisition  cycled 
They  propose  a  strategy  selection  model  integrating  judgement  and 

existing  quantitative  data  to  achieve  stated  program  objectives.  They 
first  use  a  competition  screen  to  qualitatively  screen  out  programs  not 
susceptible  to  competition  because  of  unavailability  of  technical  data, 
government  inability  to  monitor  technical  transfer,  and  other  criteria. 
Their  second  step  involves  a  cost  model  based  on  the  basic  learning 

curve  theory  where  judgement  of  shifts  and  rotations  are  made  by  program 

personnel  and  the  results  discounted  to  project  savings.  And  the  study 
points  out  in  referring  to  historical  studies: 


The  underlying  data  are  simply  too  erratic  to  permit  accurate 
projections  of  savings.  .  .  .  Also  each  successive  study 

tends  to  be  further  removed  from  the  primary  data.  .  .  . 

Another  reason  to  avoid  over-general ization  blindly  from  the 
data  base  is  that  the  cases  studied  have  had  a  conglomeration 
of  strategies,  but  the  results  have  all  been  lumped  together. ^ 


Neither  study  provides  a  theory  of  the  interaction  of  variables 
that  are  affected  by  the  competitive  forces  and  how  these  forces  affect 
the  price  of  the  various  strategies.  For  this  reason,  they  are  not 
predictive  models  of  savings. 


The  literature  contains  several  other  predictive  models,  as 
discussed  in  chapter  3,  that  were  used  in  program  specific  analyses  in 
chapter  4  to  address  one  specific  program  objecti ve--cost.  However,  I 
have  argued  that  they  have  not  empirically  demonstrated  the  validity  of 
their  models.  Moreover,  I  believe  that  there  are  so  many  interactive 
variables  that  must  be  considered  that  no  mathematical  model  can  be 
developed.  Even  if  we  could  develop  a  model  of  the  variables. 
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insufficient  data  exist  to  empirically  verify  its  validity.  Our  track 
record  of  estimating  what  the  future  cost  of  a  weapon  system  will  be, 
while  improving,  still  falls  far  short  of  acceptability.  A  quick 
perusal  of  any  quarterly  Selected  Acquisition  Report  too  vividly  reveals 
our  inability  to  accurately  project  future  weapon  systems  prices. 

Our  inability  to  accurately  forecast  future  weapons  system  costs, 
to  accurately  specify  a  model  for  savings,  or  to  empirically  verify  a 
model  because  of  a  lack  of  data--even  if  a  model  could  be  specified-- 
does  not  relieve  acquisition  personnel  of  the  requirement  to  perform 
competition  analyses  for  decision-making  and  budgeting  purposes.  The 
balance  of  this  chapter  sets  out  a  macro  model  for  use  in  competition 
analyses.  The  model  assumes  that  potential  second  sources  are 
technically  qualified  and  that  technical  data  is  available  for 
competition  at  both  the  prime  and  subcontract  level.  It  also  assumes 
that  sufficient  time  exists  to  develop  the  second  source  and  that  the 
supportabi 1 i ty  issues  have  been  examined.  These  simplifying  assumptions 
narrow  the  focus  of  the  second-source  initiative  to  that  of  achieving 
minimum  system  cost  to  execute  the  legislative  and  executive  branch 
thrusts  discussed  in  chapter  1.  The  model  does,  however,  develop  a 
framework  that  allows  the  logical  examination  of  the  impact  of  second- 
sourcing  on  other  program  objectives. ^ 


Proposed  Model 

The  proposed  model  for  use  in  competition  analyses  where 
competition  is  undertaken  for  cost-reduction  purposes  is  a  discounted 
cash-flow  investment  macro  model.  Using  this  concept,  the  government 
views  the  development  of  the  second  source  as  an  investment  decision 

where  the  nonrecurring  costs  are  incurred  by  the  government  in 

anticipation  of  a  reduction  in  future  recurring  costs.  Previous 
analyses  reviewed  in  this  study  used  the  concept  of  "savings"  as  the 
criterion  to  measure  the  cost  effectiveness  of  the  investment.  To 
arrive  at  the  measure  of  savings,  most  analyses  projected  what  the 

sole-source  price  would  have  been,  what  the  competitive  price  actually 
was,  and  claimed  the  difference  as  "savings." 

The  macro  model  replaces  the  criterion  of  savings  with  the 
criterion  of  judgement  as  to  whether  or  not  the  government  has  a  good 
probability  of  recouping  its  investment  in  nonrecurring  costs.  The 
discounted  cash  flow  investment  model  involves  three  steps:  first, 

estimating  the  additional  costs  to  the  government  because  of  second- 

sourcing,  including  the  required  nonrecurring  costs  to  qualify  the 
second  source  and  any  other  additional  costs  or  penalties;  second, 
decreasing  the  estimated  sole-source  price  by  an  amount  equal  to  those 
costs  not  susceptible  to  the  cost  pressures  of  competition;  and  third, 
making  an  informed  judgement  of  the  probability  that  the  government 
would  recover  its  nonrecurring  investment  through  reduced  recurring 
costs . 


In  contrast  to  the  studies  accomplished  to  date,  which  involved 
formal  quantitative  methods  based  on  less-than-complete  theory  and  data, 
the  proposed  method  combines  use  of  data  and  judgement.  It  does  not 
involve  the  reading  of  tea  leaves  or  a  referral  to  a  palm  reader.  It  is 
a  structured  analysis  that  uses  logic  to  iteratively  narrow  down  the 
breadth  of  available  information  and  focus  in  on  a  smaller  information 
base  where  judgements  can  be  made  on  carefully  defined  issues.  The 
model  avoids  the  drawing  of  conclusions  from  unverified  data  in  tne 
absence  of  an  adequate  theory,  and  it  avoids  the  estimating  of  several 
parameters  from  few  data  points. 

The  analysis  is  conducted  in  discounted  present  value  terms  to 
account  not  only  for  the  magnitude  of  the  cash  flows  over  time,  but  also 
for  the  timing  the  cash  flows  in  terms  of  both  costs  and  benefits. 
F.ach  potential  strategy  can  create  different  patterns  of  additional 
up-front  costs  and  downstream  henefits.  A  discounted  cash-flow  analysis 
considers  these  different  cash-flow  patterns  in  deciding  which  strategy 
is  the  most  advantageous  to  the  government.  It  is  similarly  used  in 
multiyear  and  technology  modernization/industrial  modernization 
incentives  program  analyses  where  up-front  investments  are  also  made  in 
anticipation  of  downstream  benefits. 

The  internal  rate  of  return  and  the  net  discounted  present  value 
are  two  potential  methods  to  measure  investment  potential.  The  internal 
rate  of  return  is  the  discount  rate  that  makes  the  present  value  of  the 
benefits  exactly  equal  to  the  present  value  of  the  costs.  Put  another 
way,  it  is  the  discount  rate  that  makes  the  present  value  of  the  entire 
stream  of  costs  and  benefits  exactly  equal  to  zero.  The  discounted  net 
present  value--di scussed  in  chapter  2 - - i s  the  sum  of  the  costs  and 
benefits  discounted  at  a  specified  discount  rate.  If  the  sum  is  a 
positive  number--that  is,  the  discounted  benefits  exceed  the  discounted 
costs--the  investment  should  be  made.  In  general  terms,  given  a  stream 
of  cash  flows  Cg,  Ci,  C2,  .  .  .  ,  Cn,  where  Cs  are  positive,  zero,  or 
negative,  the  net  present  discounted  value  is  given  by 

Ci  C2  Cn 

( I+r )  (1+r)2  .  .  .  ( 1 +r)n 

where  r  is  the  discount  rate.  0MB  Circular  A-94  specifies  that  a 
discount  rate  of  10  percent  be  used  where  the  cash  flows  for  government 
investments  exceed  3  years.  Therefore,  the  discounted  net  present  value 
model  is  the  appropriate  criterion  for  use  in  second-source  decision 
making. 


Non  re  cu  r  ri_ng_C  os  ts 


Exercise  of  the  3-step  macro  model  begins  with  the  relatively 
strai ghtforwa rd  process  of  estimating  the  up-front  nonrecurring  costs. 
These  readily  identifiable,  near-term  costs,  which  were  discussed  in 


chapter  2,  can  be  incurred  by  both  the  government  and  the  contractor. 
Additionally,  other  increased  costs  requiring  an  estimate  include:  the 
learning  curve  and  production  rate  penalties;  the  negative  impact  on 
other  programs  within  the  profit  center  due  to  changed  overhead 
allocation  bases;  and  the  other  impacts  discussed  in  chapter  2,  such  as 
reduced  contractor  investment  early  in  the  program  because  of 
potentially  lower  follow-on  production  contract  awards. 

Depending  on  the  requirements  of  the  technology  transfer  method,  as 
discussed  in  chapter  1,  each  potential  strategy  can  have  different  cost 
profiles,  both  in  magnitude  and  timing.  For  example,  the  technical  data 
package  transfer  would  occur  earlier  in  a  teaming  arrangement  than  it 
night  under  a  leader/fol lower  situation.  An  education  buy  might  be 
required  under  a  leader/follower  procedure  and  not  required  under  a 
teaming  arrangement.  And  learning  curve  penalties  will  vary  based  on 
the  timing  of  the  buys.  These  would  be  the  negative  "Cs"  in  our 
discounted  cash  flow  formula  and  would  be  identified  by  the  year  they 
would  be  incurred  and  the  constant  dollars  discounted  to  a  base  year. 


Costs  Not  Suscept ible  to  Competiti on  Benefits 

Step  two  involves  estimation  of  the  program  costs  that  are  not 
susceptible  to  reduction  because  of  the  pressures  of  competition. 
Included  would  be  material  and  subcontract  purchases  already  competed  by 
the  developer  and  contracted  for  on  a  firm-fixed-price  basis.  The 
percentage  of  total  contract  expenditures  fitting  this  category  can  vary 
significantly  between  contracts.  Table  5-1  illustrates  how  these 
figures  varied  on  12  programs.  Competitively  purchased  material  as  a 
percentage  of  total  contract  costs  ranged  from  2.1  percent  to  50.2 
percent.  The  study  that  compiled  these  data  also  concluded  that 
42.94  percent  and  78.93  percent  of  all  subcontract  dollars  were 
spent  competitively  by  DOD  contractors  and  government-owned,  contractor- 
operated  plants,  respectively. 

Also  included  in  costs  not  susceptible  to  competition  benefits  are 
those  material  and  subcontract  purchases  where  a  potential  second  source 
would  use  the  same  supplier  as  the  developer  for  schedule  availability, 
technical  capability,  or  economic  production  quantity  reasons.  This 
also  holds  true  for  the  following  costs: 

t  When  there  are  fully  automated  manufacturi ng  processes. 

0  When  there  are  mature  components  where  no  potential  learning 

exists  as  verified  by  a  fully  validated  work  measurement  system. ^ 

0  When  the  components  are  sold  in  substantial  quantities  to  the 

general  public  with  an  established  catalog  or  market  price. ^ 

0  When  the  prices  are  otherwise  set  by  law  or  regulation. 

0  When  mandatory  sources  are  specified.^ 
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FIREFINDER  data  reflects  expected  costs. 

$  not  avai lable. 

Major  sub  controls  62%  of  contract  material  cost  of  which  19.1%  is  competitive. 
Competition  data  based  on  sample  of  purchase  orders. 


•  When  there  are  any  other  costs  that,  in  the  opinion  of  the 
program  manager,  are  not  susceptible  to  cost  reduction  through  the 
direct  pressures  of  competition. 


Finally,  allocations  for  independent  research  and  development,  and 
facilities  capital  cost  of  money  should  be  identified  as  being  in  the 
long-term  best  interests  of  the  government  and  tallied  as  not  being 
susceptible  to  reduction  through  competitive  pressures. 

Those  costs  not  susceptible  to  reduction  through  the  pressures  of 
competition,  or  those  costs  whose  reduction  is  not  in  the  best  long-term 
interests  of  the  government  would  be  subtracted  from  the  estimated 
sole-source  price  in  the  year  they  would  be  incurred,  leaving  a 
projected  dollar  figure  for  each  year  of  an  amount  susceptible  to  the 
pressures  of  competition.  These  constant  dollar  figures  would  be 
discounted  to  the  base  year. 


Judgement  F actors 

Step  3  involves  calculating  the  percentage  reduction,  in  net  present 
value  terms,  that  total  costs  susceptible  to  competition  would  have  to 
be  reduced  to  recover  the  up-front  investment  costs  and  added  costs  to 
this  and  other  programs.  This  is  accomplished  by  dividing  the  sum  of 
estimated  nonrecurring  and  other  costs  by  the  recurring  costs 
susceptible  to  the  pressures  of  competition.  An  informed  judgement 
could  then  be  made  by  the  program  manager  of  the  probability  that  the 
recurring  costs  susceptible  to  competition  pressures  could  be 
sufficiently  reduced  to  recover  the  investment  costs  and  penalties. 
Both  figures  would  he  expressed  in  constant  dollars  discounted  at  10 
percent  in  accordance  with  0MB  circular  A-94.  Factors  to  consider  in 
establishing  the  subjective  probability  include  contractor-related 
issues  and  program-specific  characteristics. 


_ Issues.  Traditional  microeconomic  theory 

assumes  firms  want  to  maximize  profits.  However,  others  have  argued 
differently.  William  J.  Baumol  believes  that  large  firms  attempt  to 
maximize  sales  rather  than  profits.^  And  Richard  Cyert  and  James  March 
posit  that  firms  simply  attempt  to  "satisfice"--reach  some  satisfactory 
level  of  multiple  goal  accomplishment:  profit,  sales,  market  share, 
production  and  inventory,^  Peter  Drucker  also  comments  on  corporate 
objectives  and  states  that  the  long-run  objective  is  survival  and  that  to 
emphasize  only  profit  misdirects  managers  to  where  they  may  endanger  the 
survival  of  the  business. 


The  Army  Procurement  Research  Office  conducted  a  1981  study  on 
government  contractor  motivation.^  Its  research  concluded  that 
industry  felt  its  objectives  were,  in  order  of  priority,  to  provide  a 
good  product,  to  maintain  a  long-term  continuing  government 
'■elationship,  to  improve  cash-flow,  to  make  a  profit,  to  develop  new 
capabilities,  to  maintain  a  positive  public  image,  and  to  use  excess 
capacity.  And,  as  could  be  expected,  this  list  varied  by  both  firm 


size  and  growth  status,  manufacturing  process  used,  and  the  type  of 
industry.  One  of  the  major  conclusions  was  that  the  government  must 
understand  contractor  motivation  in  order  to  give  proper  incentives  fo r 
contractor  performance.^ 

Others  have  written  about  what  might  happen  when  split  awards  are 
made  during  production  competition.  Brent  Meeker,  writing  in  Program 
Manager,  argues  that  any  scheme  that  <ioes  not  provide  for  zero 
allocation  can  engender  reverse  competition,  a  situation  where  both 
companies  decide  that  the  smaller  quantity  is  sufficient,  bidding  higher 
prices  accordingly.^ 

The  major  point  of  these  discussions  and  examples  is  that  the 

contractor  might  have  objectives  other  than  profit  maximization.  These 
can  include  use  of  idle  capacity  or  the  development  of  new  technology. 
The  program  manager  must  be  sensitive  to  these  motivations  in  his 
subjective  probability  assessment  of  how  much  the  developing  contractor 
can  and  will  cut  prices. 

Prog ram-Spec i fic  Charact eristics.  Several  program  characteristics 
should  be  considered  by  the  program  manger.  These  include  the  total 
quantity  subject  to  competition,  maturity  of  the  product  design, 

estimates  of  the  efficiency  of  the  developer,  amount  of  touch  labor,  and 
the  amount  and  type  of  subcontracting. 

The  total  quantity  subject  to  competition,  including  foreign 
military  sales  and  other  systems  using  identical  components  or 

production  processes  and  facilities,  affects  the  opportunity  to  recoup 
nonrecurring  costs.  Generally,  the  larger  the  production  quantity  and 
the  longer  the  manufacturing  horizon  of  this  production  quantity,  the 
greater  the  chance  the  pressures  of  competition  can  exert  their  downward 
price  pressures.  This  results  because  the  contractors  have  more 
opportunity  to  efficiently  plan  their  production  processes.  Moreover, 
additional  incentives  to  invest  in  efficient  equipment  might  occur  if  it 
helps  achieve  their  corporate  objectives. 

The  maturity  of  the  product  design  affects  the  number  of 

engineering  changes  that  might  occur  during  weapon  system  production. 
This  in  part  determines  the  ability  of  the  contractor  to  reduce  the 
quantity  of  systems  engineering  and  program  management  costs  charged  to 
the  program,  usually  a  level -of-effort  charge  partially  dependent  on  the 
maturity  of  the  system.  It  is  also  a  factor  in  determining  the 
magnitude  of  interface  charges  the  developing  contractor  will  nave  with 
the  second  contractor.  Additionally,  it  will  determine  the  incremental 
configuration  management  costs  necessary  to  keep  the  technical  data 
package  updated. 

Subjective  evaluations  of  the  efficiency  of  the  contractor  and  toe 
quantity  of  touch  labor  and  type  of  jobs  they  perform  will  help  in 
judging  the  probability  of  cost  reduction.  Particularly  useful  is  an 
evaluation  of  the  contractor's  use  of  a  work  measurement  sy stem--such  as 
MIL-STD-1567 .  Additionally,  the  use  of  the  MIL-Q-9858A  qualitv  system 


reports  on  the  costs  of  quality,  scrap,  and  rework  are  enlightening  on 
the  efficiency  of  production  methods.^ 

The  amount  and  type  of  subcontracting  is  related  to  the  complexity 
of  the  product.  If  the  complexity  of  subcontracts  is  high,  or  there  is 
only  one  supplier  or  subcontractor  for  much  of  the  required  materiel, 
the  chances  for  price  reduction  are  reduced.  The  state  of  the 
contractor's  purchasing  system  as  revealed  in  the  government's 
"contractor  procurement  system  review"  is  a  good  gauge  of  contractor 
attempts  to  stimulate  subcontract  competition.  Additionally,  a  review 
of  the  program  specific  make-or-buy  plan  would  also  give  insight  into 
the  potential  for  price  reduction. 

Taken  together,  consideration  of  these  areas  that  affect  the 
potential  for  competitive  cost  reduction-company  and  program-specific 
f actors--wi 1 1  help  the  program  manager  determine  the  probability  of 
recouping  the  investment  in  nonrecurring  costs.  In  many  ways  this 
logical  thought  process  is  similar  to  a  "should  cost"  study.  And  even 
if  it  is  judged  by  the  program  manager  that  the  costs  cannot  be 
recovered,  the  information  derived  from  the  analysis  puts  the  contract 
negotiator  in  a  position  to  negotiate  better  prices  in  the  sole-source 
situation. 


Sty  1 i zed  Example 

A  stylized  example  can  serve  to  illustrate  the  exercise  of  the 
model.  All  dollar  figures  are  discounted  present  value. 


Gi ven: 

Nc  =  nonrecurring  cost  =  $200 

Q  =  total  quantity  =  50 

P  =  projected  sole  source  price  =  $1500 

R  =  rate  and  learning  curve  penalty  =  $250 

C  =  projected  costs  not  susceptible  =  $100 

Step  1 

Added  costs  =  Nc  +  R 

$200  +  $250  =  $450 


Step  ? 


Costs  susceptible  to  competitive  pressures  =  P  -  C 
$1,500  -  $100  =  $1,400 
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Step  3 


Needed  percentage  reduction  = 

Added  costs  r-  costs  susceptible  to  competitive 
pressures  =  $450  -r  $1  ,400  =  .32  x  100  =  32  percent 


In  the  example,  recurring  costs  susceptible  to  competition  would  have 
to  be  reduced  32  percent  to  recoup  the  investment  and  other  added  costs. 
If  that  required  percentage  were  higher--say  50  percent--it  would  be 
less  likely  that  recurring  costs  could  be  reduced  by  that  amount.  In 
either  case,  the  program  manager  must  make  a  decision  on  the  probability 
that  costs  susceptible  to  the  pressures  of  competition  can  be  reduced  by 
the  derived  percentage  figure. 


Sensitivity  Analysis 

Having  used  the  proposed  framework  in  a  logical  manner  to  d^aw 
reasoned  conclusions,  analysts  should  conduct  a  sensitivity  analysis  to 
determine  what  the  impact  of  changes  in  key  factors  is  on  the 
conclusions.  It  serves  as  a  "what  if"  exercise  and  provides  additional 
information  to  the  decision  maker  about  the  impact  of  uncertainty  on  his 
judgmental  decision. 

Factors  to  consider  include  firmness  of  the  estimated  quantity, 
changes  in  the  inflation  rate,  certainty  of  estimates  of  the 
nonrecurring  costs,  changes  in  the  timing  of  the  buys,  or  changes  in  the 
timing  the  second  source  becomes  a  viable  suppler.  Reduced  quantities 
would  dictate  that  the  percent  reduction  of  the  sole-source  price  on 
remaining  quantities  would  have  to  be  higher  to  recoup  nonrecurring 
costs.  Slipping  production  quantities  into  future  years  would  reduce 
the  present  value  of  the  out-year  benefits.  Delaying  the  tooling  of  the 
second  source  would  reduce  the  present  value  of  the  nonrecurring  costs. 

F.ach  change  in  the  calculation  that  affects  either  the  timing  of 
the  investment  or  the  timing  of  the  returns  affects  the  price  change 
needed  to  recoup  the  investment  cost.  And  the  proposed  macro  model 
pertains  only  to  the  program  objective  of  cost  control.  But  the  program 
objectives  are  interactive.  For  example,  delaying  the  development  of 
the  second  source  reduces  the  cost  of  the  program  in  discounted  dollars. 
However,  this  action  also  affects  the  availability  of  the  weapon  system. 
And  if  the  original  production  schedule  is  maintained,  bringing  the 
second  source  on-line  later,  the  opportunity  to  recoup  the  nonrecurring 
costs  is  decreased  because  of  the  reduced  quantity  subject  to 
competition.  The  analysis,  while  developed  to  focus  on  the  cost 
objective,  provides  a  focus  on  the  trade-offs  between  program  object i ves . 
Other  benefits  that  could  come  from  the  analysis  would  be  a  more 
informed  judgement  on  the  costs  of  an  enhanced  surge  capability. 
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Recomine  nrlatio  n  s 

Two  recommendations  are  made  to  Headquarters  AFSC: 

(1)  Require  competition  analyses  based  on  an  investment  recoupment 
model  in  lieu  of  a  projected  savings  model  based  on  historical  data. 

(2)  Require  competition  analyses  in  discounted  dollars  in 
accordance  with  0MB  Circular  A-94. 


Recommendation  One 

This  recommendation  is  made  to  encourage  analysts  to  move  away  from 
the  concept  of  projecting  savings  because  of  the  problems  of  methodology 
and  data  discussed  in  this  report.  It  is  also  made  because  of  a  concern 
that  budget  decrements  could  occur  through  a  directive  to  compete  a 
system  based  on  the  faulty  assumption  that  a  specified  percentage  savings 
will  result  through  competition.  Three  actions  wi_l_l_  institutionalize 
this  recommendation. 

First,  Air  Force  regulations  dictate  guidelines  in  this  area. 
AFSC /AC  should  initiate  recommendations  to  modify  the  following  Air 
Force  regulation  and  pamphlet: 

AFR  178-1,  paragraph  1-lh.  Require  that  economic  analysis  within 
the  meaning  of  the  regulation  be  accomplished  for  major  weapons  systems 
procurements  on  the  costs  and  benefits  of  competition  in  production. 
The  regulation  should  require  that  explicit  recognition  be  given  to  all 
costs  and  benefits  to  the  government  as  discussed  in  this  study,  taking  a 
life-cycle  perspective  of  the  given  program.^ 

AFP  178-8,  Section  11.  Incorporate  a  sample  economic  analysis  in 
the  document  using  the  framework  for  second-source  production 
competition  analysis  set  out  in  this  study. ^ 

Second,  training  courses  for  program  control  and  cost  analysis 
personnel  should  incorporate  materials  on  this  subject  in  their 
curricula  to  clarify  the  shortcomings  of  existing  studies  and  present  a 
reasoned  alternative.  Additionally,  the  Defense  Systems  Management 
College  is  preparing  a  proposed  DOD  Competition  Handbook  which  discusses 
previous  shortcomings,  proposes  discounting,  and  also  has  a  caution 
about  the  use  of  historical  data.  When  printed,  distribution  should  be 
made  to  AFSC  personnel. 

Third,  top-level  DOD  personnel  must  avoid  references  to  average 
savings  in  their  speeches,  articles,  or  testimony  before  the  Congress, 
so  that  they  do  not  perpetuate  the  faulty  conclusions  about  average 
s  a  v  i  n  a  > . 


122 


Recommendation  Two 

The  second  recommendation  originates  because  of  the  time  value  of 
money.  Again,  an  Air  Force  regulation  mandates  procedures.  AFSC/AC 
should  inititate  a  recommendation  to  change  AFR  1/3-11,  paragraph  7J,  to 
require  discounting  of  cash  flows  for  ICA  estimates  of  the  impact  of 
competition  in  production.^ 

Additionally,  AFSCR  173-9,  table  3,  rule  2,  should  be  changed  to 
include  the  following  questions:^ 

Are-- 

Production  competition  cash-flow  estimates  discounted? 

Were-- 

Ru 1 es-of -thumb  or  unweighted  average  savings  estimates  for 
the  impact  of  production  competition  avoided? 


Closi n g_  Thoughts 

Competition  and  the  free  enterprise  system  have  helped  give  this 
country  the  highest  standard  of  living  known  to  man.  However,  in  many 
instances  the  competitive  forces  of  the  market  place  are  absent  in  the 
defense  market  for  major  weapons  systems;  there  exists  only  one  buyer, 
and  the  barriers  to  entry  into  the  market  preclude  many  firms  from 
entering.  These  barriers  include  both  the  high  cost  of  investment  and 
the  lack  of  availability  of  technology.  Developing  a  second  source  at 
government  expense  is  one  option  to  stimulate  competition.  However,  as 
seen  in  the  subcontract  example,  not  all  dollars  placed  "noncompet- 
itively"  at  the  prime  contractor  level  are  spent  noncompetiti vely. 

There  are  other  options  to  reduce  the  high  cost  of  weapons 
s.ystems--breakout,  multiyear  procurement,  technology  moderni zat  i on ' 
industrial  modernization  incentive  program  ( 1 M I P ) ,  and  increased 
subcontract  competition.  The  use  of  these  mechanisms  to  achieve  cost 
avoidances  should  also  be  explored.  They  can  affect  program  cost 
without  the  necessary  investment  in  duplicate  tooling.  And  in  the  case 
of  the  IMIP  program  the  benefits  of  automated  production  can  be  reaped 
on  several  programs  by  tooling  up  the  system  developer  after 
down-selecting  at  the  beginning  of  full-scale  engineering  development. 

Second-sourcing  in  production  is  a  viable  tool  for  cost  reduction, 
but  neither  a  total  virtue  nor  original  sin.  It  is  a  tool  to  be  used  to 
achieve  specific  objectives.  It  will  not  always  save  money  as  some  have 
intimated.  Given  the  appropriate  conditions,  it  can  save  money  if  the 
recurring  cost  reductions  exceed  the  nonrecurring  costs.  The 
methodology  proposed  in  this  report  can  help  make  this  decision  by 
elevating  to  a  visible  level  critical  information  necessary  to  make 
appropriate  judgements.  And  even  if  it  is  determined  that  it  is  not 
economically  feasible  to  stimulate  competition,  the  program  manager  is 
armed  with  information  to  have  his  staff  better  prepared  to  negotiate 
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